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ABSTRACT 



Guidelines are presented to help managers estimate the costs of 
computer programming. The guidelines summarize a statisticstl 
ajialysis of I69 computer programming efforts with equations to 
estimate man months, computer hours, and months elapsed, and 
eilso planning factors such as man months per thousajid instructions. 
Opinions, rules of thumb, ajid experience data based upon literature 
search ajid experience supplement the statisticeil results. Forms 
with the guidelines are organized into six sections corresponding 
to a six-step division of the computer programming process. Advice 
is given on the integration of cost estimates into a cost analysis 
to justify and plan ADP projects. 



iii 



CONTENTS 

Foreword ii 

Abstract iii 

List of Figures vi 

List of Tables vii 

I. Introduction 1 

Purpose 2 

Scope k 

Background and Sources of Data k 

Cost Estimation Methods 5 

Approach of the Handbook 7 

Interpretation 11 

II, Using the Handbook I3 

Structure of the Handbook I3 

Preliminary Cost Evaluation Analysis I9 

Supplementing the Handbook 29 

III. Preliminary Planning and Cost Evaluation 33 

Activity Definition 33 

Computer Programming Cost Factors 36 

Additional Comments on Selected Cost Factors 38 

Planning Factors 39 

IV. Information System Analysis and Design kl 

Activity Definition kl 

Computer Programming Cost Factors k^ 

Additional Comments on Selected Cost Factors kQ 

Planning Factors kg 



iv 



V. CcMputer Program Design, Code, and Test 51 

Activity Definition 51 

Computer Programming Cost Factors 5^ 

Additional Comments on Selected Cost Factors 6o 

Planning Factors 65 

Computer Programming Cost-Estimating Equations 74 

VI. Information Processing System Integration Test 91 

Activity Definition 9I 

Computer Programming Cost Factors 93 

Planning Factors 96 

VII. Information System Installation and Turnover 97 

Activity Definition 97 

Computer Programming Cost Factors 100 

Additional Comments on Selected Cost Factors IQli 

Planning Factors I05 

VIII. Computer Program Maintenance 107 

Activity Definition IO7 

Computer Programming Cost Factors 110 

Additional Comments on Selected Cost Factors ll4 

Planning Factors II5 

Glossary A. Definitions of Variables II9 

Glossary B. Definitions of Symbols I32 

Glossary C. Use of Terms I33 

Glossary D. Definition for the Code to Rate the 135 
Impact of Cost Factors 

References I3Y 



LIST OF ILLUSTRATIONS 

FIGURE 

1 Computer Programming Project Cycle 6 

2 Recycling the Computer Programming Process Steps 9 

3 Sample Activity Definition Format ik 
k Sample Computer Programming Cost Factors 15 

5 Sample Planning Factor Format 17 

6 Sample Estimating Equations Format l8 

7 Sample Cost Justification Form 21 

8 Sample Project Description Form 24 

9 Sample Budget/Schedule V/ork Sheet 26 
10 Sample Budget/schedule Summary 28 



VI 



LIST OF TABLES 



TABLE 



I. 



II. 



Activity Definition — Preliminary Planning and Cost 
Evaluation 

Computer Programming Cost Factors — Preliminary Planning 
and Cost Evaluation 



III. Planning Factors — Estimation of Tasks Within Activity 

IV. Activity Definition — Information Processing System 
Analysis and Design 

V. Computer Programming Cost Factors — Information 
Processing System Analysis and Design 

VI. Planning Factors — Estimation of Tasks Within Activity 

VII. Activity Definition for Computer Program Design, Code, 
and Test 

VIII. Computer Programming Cost Factors — Computer Program 
Design, Code, and Test 

IX. Planning Factors — Quotes Without Comment 

X. Planning Factors— Resource Rate per 1000 SOURCE 

Instructions 

XT. Planning Factors — Resource Rate per 1000 OBJECT 
Instructions 

XII. Planning Factors — Total Pages of Documentation per 
1000 OBJECT Instructions 

XIII. Planning Factors — Man Months Expenditure Rates 

XIV. Planning Factors — Estimation of Tasks Within Activity: 
Unit Cost 

XV. Planning Factors — Estimation of Tasks Within Activity: 
Percent of Other Item 

XVI. Planning Factors — Program Development Computer Time 

XVII. Planning Factors — Estimating Computer Hours from 
Man Months 



33 
36 

39 
Ui 

^5 

k9 
51 

5^ 

65 

67 
68 

69 
TO 

71 

72 
73 



VI 1 



TABLE 

XVIII. Comparison of Estimating Equation Characteristics j6 

XIX. Equation for Estimating Man Months: Total Sample 77 

XX. Equation for Estimating Computer Hours: Total Sample 78 

XXI. Equation for Estimating Months Elapsed: Total Sample 79 

XXII. Equation for Estimating Man Months: Medium Computer 80 

XXIII. Equation for Estimating Months Elapsed: Medium Computer 8I 

XXIV. Equation for Estimating Man Months: Large Computer 82 

XXV. Equation for Estimating Months Elapsed: Large Computer 83 

XXVI. Equation for Estimating Computer Hours: Other Subsample' &k 

XXVII. Equation for Estimating Computer Hours: Utility Subsample 85 

XXVIII. Equation for Estimating Months Elapsed: POL Subsample 86 

XXIX. Equation for Estimating Maji Months: MOL Subsample 87 

XXX. Activity Definition — Information Processing System 91 
Integration Test 

XXXI. Computer Programming Cost Factors — Information Processing 93 
System Integration Test 

XXXII. Planning Factors — Costing System Integration Test 96 

XXXIII. Activity Definition — Information Processing System 97 
InsteGLlation and Turnover 

XXXIV. Computer Programming Cost Factors 100 

XXXV. Planning Factors — Costing Demonstration Test 105 

XXXVI. Planning Factors — Estimation of Tasks Within Activity I06 

XXXVII. Activity Definition — Computer Program Maintenance 107 

XXXVIII. Computer Programming Cost Factors — Computer Program 110 

Maintenance 



viii 



TABLE 



XXXIX. Planning Factors — Estimating Nijmber of Maintenance 
Programmers 

XL. Planning Factors — Estimating Progrsun Maintenance 
Computer Hours : Unit Cost 

XLI. Planning Factors — Estimating Program Maintenance 
Computer Hours: Unit Cost 

XLII. Planning Factors — Estimating Prograjn Maintenance 
Computer Hours: Percent of Other Item 



115 
116 

117 
118 



ix 



SECTION I 
INTRODUCTION 



This Handbook is an organized collection of material intended to help managers 
estimate the cost of computer program development. Specifically, both quanti- 
tative and qualitative guidelines are given for estimating the resources, i.e., 
man months, computer hours, and months elapsed, to be used in conducting the 
six activities that are assiimed in this Handbook to constitute the computer 
programming process. This division of the process forms the basis for the 
organization of the Handbook and includes the following sections: Preliminary 
Planning and Cost Evaluation; Information System AneLIysis and Design; Computer 
Program Design, Code, and Test; Information System Integration Test; Information 
System Installation and Turnover; and Computer Program Maintenance. In addition 
to a brief description of the activity in terms of tasks, inputs, and outputs, 
each of these sections includes a list of cost factors together with some 
indication of their influence on costs and planning factors such as production 
rates, e.g., man months per 1000 instructions and comparisons between the costs 
of the particxilar activity to total process costs. Reflecting the infant state 
of the art in planning for computer prograimning, the guidelines presented for 
most of the programming activities are mostly qualitative, based upon the 
authors* experience or upon data eind opinions found in the technical literature. 
An exception to the general dearth of quantitative data in the Handbook, the 
section on Computer Program Design, Code, and Test presents the results of a 
statistical analysis of numerical data on costs and cost factors that describe 
169 completed prograimning projects. This section, in addition to the quanti- 
tative planning factors, includes equations to estimate the resources needed 
for conducting the program design, code, and test activity. Several sets of 
equations were derived — based upon data from entire sample as well as data 
from subsamples to identify the type of programming languages used (machine- 
oriented language versus procedure-oriented language), the type of application 
(business, scientific, software, and other), and the type of computer (small, 
medium, and large--based upon cost). 

It is assumed that one use of the cost estimates for computer programming is 
to help managers decide whether to proceed with the implementation of plans 
for the computer program being costed. To aid the manager in orgajiizing the 
data for a cost evaluation, the section on Using the Handbook contains examples 
of forms for recording cost estimates to compare alternatives. The section on 
Preliminary Planning and Cost Evaluation contains information on the factors 
that affect the cost of planning and making the estimate itself. 

This Handbook does not compare and evaluate the various guidelines presented. 
Use of all the applicable guidelines for ajiy particular job woxild iindoubtedly 
yield a niimber of estimates with a wide variation. Even the resxxlts derived 
by statistical analysis do not provide highly accurate estimating relationships. 
Therefore, this Handbook shoxild be interpreted as an early effort to collect 



and systematize some of the available knowledge on cost estimation for computer 
programming. As such, it should be regarded by the user as a supplement to 
managerial judgment rather than a replacement for it. 

Further, we recommend that, wherever possible, the users of the Handbook sup- 
plement the numerical data provided by recording data on costs and cost factors 
on their own computer programming work and by analyzing these data to obtain 
improved guidelines. 

The remainder of this Introduction clarifies the purpose of the Handbook, 
defines the scope of the material, identifies the sources of data, describes 
the methods of estimation, introduces the division of the computer programming 
process, and presents the caveats on data accuracy and reliability. The next 
section, Using the Handbook, explains (l) the various forms and tables that 
display the data, (2) how to use the data in preparing a cost evaluation, and 
(3) the ways in which the user could supplement the Handbook. 

1. Purpose . IBM's expected $200 million investment in software for the 
System 36O computer series and their much-publicized difficulties in delivering 
many of the computer programs dramatically highlight the lack of effective 
tools for controlling and planning computer programming (26), Managers at IBM 
are not alone; underestimates for costs and lead times are the rule rather than 
the exception in the teeming new industry of computer programming. One reason 
for this situation is that relatively little has been done to record, system- 
atize, and generalize technical management experience. Meanwhile, the number 
of new machine installations and new applications continues to grow rapidly, 
demanding new managers--often inexperienced. At the same time, more and more 
experienced managers are swallowed up in the ambitious frontier-breaking 
ventures such as large military or space systems with major ADP subsystems, 
large integrated time-sharing networks for government and industry, and 
development of computer programs (software) for new computers. The void of 
organized knowledge and formal material for the technical management of computer 
programming dictates that most learning now and in the future probably will be 
accomplished by personnel obtaining actual experience in the field. 

But there are some efforts under way to accumulate, integrate, and convert 
experience into useful guidelines. Among these efforts is the Programming 
Management Project at System Development Corporation (SDC), whose members 
developed this Handbook under a contract with the Air Force Electronic Systems 
Division, Deputy for Engineering and Technology, Directorate of Computers. 

The major purpose is to begin to provide the operating manager of computer 
programming projects with a methodology and the data that can help him forecast 
the resources required and incorporate these estimates into cost evaluation 
studies, broader project plans, and cost control systems. To these ends, the 
available data are organized in consistent formats, and guidelines are given 
for how to use these data in preparing a cost evaluation for a proposed 
computer programming effort. 



A second purpose of the Handbook is to encourage the manager to collect and 
analyze data on costs and cost factors from his own operations. The formats 
themselves provide both a way and an incentive for an estimator to supplement 
the data presented. These additions to the material are needed for several 
reasons : 

As indicated earlier, the only section in the Handbook with a repre- 
sentative sample of numerical data is that on Computer Program Design, 
Code, and Test. To improve the usefulness of the other sections, 
numerical data should be collected and analyzed to derive planning 
factors and estimating equations. 

. Even the estimating relationships provided in the section on Computer 
Program Design, Code, and Test, although based upon statistical analysis 
of a reasonably-sized sample, still have large standard errors. 
Therefore, ve expect new estimating relationships based upon collection 
of local data and their analysis in an individual organization might be 
more accurate than those derived in our statistical analysis because 
many of the sources of cost variation would be eliminated. That is, 
many of the factors identified as variables in this Handbook such as 
the type of computer application and personnel would be held relatively 
constant in a particiilar organization or installation. 

• Finally, new data will be needed to accurately reflect the changing ADP 
technology. Change is the predominant characteristic of the world of 
computers, and in this, computers with features such as multiprocessors 
with logarithmically increasing speeds have been the forerunners. 
Applications that feature networks with automatic inputs and outputs 
and time-sharing capabilities highlight the transition in computer 
configurations. New programming techniques such as data management 
systems, query languages, programming languages, and compilers are 
also a significant part of the change. Such changes have a marked 
impact on the economics of data processing generally, and consequently 
on the estimation of programming costs. The principles of estimation 
will not change; the basic ways of arriving at a cost estimate today 
will be the same. But the data, and the cost factors and equations 
developed from them, will change, i.e., the material in this Handbook 
will become obsolete. 

Therefore, a Handbook such as this, to remain useful, must be dynamic. It 
requires addition to make it more complete and accurate, and continual updating 
to reflect changes such as new compilers, machines, training methods, or 
programming languages. And so, this volume, designed as a Handbook, may take 
on more of the characteristics of a workbook— periodically updated and continually 
improved . 



2. Scope . We confined the scope of this Handbook to the estimation of the 
costs in terms of resources required in the ccxnputer programming process and 
the organization of these cost estimates for management review. The broader 
but related problems of planning and control are not directly addressed. 

We also do not address narrower subjects that are directly related to costs 
such as selection, training, and appraisal of computer programmers (35; ^6, ks) 
equipment costs, ^ and the associated question of purchase versus lease {9, 5^) 
Prices to convert the estimates of basic resource units (man months, computer 
hours, elapsed time) to dollar costs are not provided. ^ 

This Handbook is intended as a concise reference covering the particular subject 
of computer programming cost estimation. It is not a treatise on the subject, 
but a compilation of facts and authoritative opinion. Wherever possible, we 
used charts or tables in place of prose. For this reason, most of the prose is 
found in this Introduction and in the next section on how to use the Handbook. 

3. Background and Sources of Data . The Handbook summarizes the results of a 
major task in the Programming Management Project at SDC— to derive equations 
for estimating the costs of the computer program design, code, and test activity 
in computer programming. To conduct this task. Project members used a question- 
naire to collect numerical data that measure costs and cost factors for completed 
programming efforts and subsequently analyzed these data using statistical 
methods. These analyses were done in cycles; each succeeding cycle, aimed at 
improving earlier results, corresponded to the collection of new numerical data 
and their subsequent analysis. In the first cycle, data for 27 programming 
efforts (data points) completed at SDC were analyzed and the results were 
reported in the fall of 196k (19). The work in the second cycle, an analysis 

of a total of T^ data points also representing SDC programming work, was 
reported in detail in the fall of I965 (62), and summarized and extended in the 
spring of I966 (53). The third cycle, which included analysis of I69 data 
points, 3 69 of which were programmed at SDC and 100 at Air Force and industrial 
programming organizations, provides most of the quantitative guidelines found 
in the section on Ccxnputer Program Design, Code, and Test. 



For prices of various items of equipment, physical characteristics and opera- 
tional characteristics including the performance of certain configurations on 
benchmark problems, see (5) There have also been some attempts to develop 
cost estimating relationships for new computer hardware for special 
applications (29). 

For salary ranges for different job categories, see (17). 

•^This data base, and the statistical methods used in the analysis, are 
summarized in (20). 



In addition to the results of the statistical analysis, the contents of this 
Handbook were obtained from two other sources of readily available material: 

a. The general published literatiire in the files of the Programming 
Management Project. Primarily, periodicals were reviewed. An extensive 
search of all the available literatiire could not be made. The references 
used are listed at the end of this volume. 

b. The technical knowledge and accumulated experience of members of the 
Programming Management Project at SDC. All statements that are not spe- 
cifically referenced should be considered the best judgment of the Project 
members. 

h. Cost Estimation Methods . The data and guidelines in the Handbook can be 
used in various ways to make a cost estimate. In using these methods, an 
analyst must infer some sort of relationship between the unknown future costs 
and accumulated past experience (39) • There are essentially four methods by 
which this can be done : 

a. Specific analogy , where costs for a new item are estimated by using 
the known costs for a similar item produced earlier. 

^* Unit price , where the cost of a new item is estimated 
by the product of the number of units to be delivered in the new 
item (e.g., number of instructions) and previously determined 
cost per unit. 

c. Percent of other item , where the cost of a new item is 
estimated as a predetermined percent of the cost of another 
item, e.g., the cost of Computer Program Design, Code, and Test 
might be a fixed fraction of total computer programming costs. 

d. Parametric equations , where the cost of the new item is estimated 

from an equation which is a function of various characteristics of the require- 
ments for the item resources expected to be used and working conditions. 

Each of these methods is comparative; each is dependent upon an applicable data 
base from past experience, each assumes that the past is prologue. To estimate 
a particular job, each method may be used alone, or in combination with the 
others. They also may be applied at various levels of aggregation; e.g., to 
estimate computer programming costs, the total computer program may be estimated 
as a whole, or broken up into components, such as steps in the programming 
process. All estimates of costs, no matter how subjective they may appear to 
be, are actually based on one or more of the above four methods. 

In the sections of this Handbook, corresponding to six activities or steps in 
computer programming, we present data to enable the user to make estimates using 
the last three methods cited: unit price, percent of other cost, and parametric 



planning 
factors 



equations. The first two of these are called Planning Factors In the text. To 
use the specific analogy technique, some effort is under way to develop an 
index of ADP system development efforts in the Air Force (23)* Ideally such 
an index would Identify and help retrieve histories of completed projects that 
were similar to a proposed ADP system. The costs of the completed efforts 
could then be used to estimate the new project by the specific analogy method. 

5. Approach of the Handbook . In the Handbook, we have divided the programming 
process into distinct steps, or activities, for planning and estimating purposes. 
From the technical manager *s viewpoint, there are two reasons for breaking up 
the programming process into steps. Different steps may represent fundamentally 
different kinds of jobs with consequently different cost implications, such as 
different types of personnel, tasks, and/or locations. Also, If the completion 
of a step can be a clearly defined and identifiable event with a definitive 
end product, these events constitute milestones useful for control. 

The steps making up the computer programming process, or project cycle, are 
assiomed to be : 

a. Preliminary Planning and Cost Evaluation 

b. Information System Analysis and Design 

c. Conputer Program Design, Code and Test (production) 

d. Information System Integration Test 

e# Information System Installation and Tiornover 
f . Computer Program Maintenance 

These steps, with their principal outputs, are illustrated in Figure 1; a more 
detailed description of the tasks and outputs of each step is provided in suc- 
ceeding sections. In this Handbook, we regard an information (processing) 
system as possibly containing many components or subsystems such as computers, 
computer programs, communications, displays, and operational procedures, all 
designed or tailored and used to work together to help an organization perform 
one or more missions. A large (high cost) computer programming effort usually 
corresponds to the development of a complex information system in which most, 
if not all, subsystems and components would be new or changed. In this case, 
all the subsystem developers, for example, the computer contractor, would carry 
on a set of activities similar to those listed above. In the simplest kind of 
system development, frcxn the viewpoint of the computer programming organization, 
all of the components or subsystems would remain relatively fixed, e.g., same 
computer, and the system analysis and design would be minimal, with the 
remainder of the activities confined to the computer program as a single 
component . 



Using the earlier reasoning that identification of more tasks and monitoring 
of these will help improve management, even more steps or divisions would be 
desirable for control and planning. Previous work by the Programming 
Management Project and others on the planning problem in computer programming 
has used even finer divisions for the process (6, l8, 30) . However, the 
absence of cost data would make further division of the process in this 
Handbook a useless exercise. Many times, for planning purposes, the tasks of 
Program Design, Coding, and Testing are separated, but in the Handbook they 
are grouped together as a single activity because the numerical data used in 
the analysis were gathered in that way. The other five activities corresponding 
to the remaining divisions were identified for several reasons: 

To stimulate recognition of a complete spectrum of computer programming 
work in planning that involves cost forecasts. Even without large 
amounts of numerical data, a manager may avoid underestimates if he 
recognizes that each of the activities may involve a major expenditure 
of manpower or lead time. 

. To stimulate planning and cost comparison work in computer programming 
by identifying a specific activity for such work as part of the process. 

To recognize and acknowledge the trend toward larger integrated 
information systems by identifying the potentially costly activities 
that follow computer program design, code, and test, e.g., system 
integration test. The need to identify, plan for, and cost such 
activities is seen more clearly for large information systems because 
the major subsystems and components in a large information system are 
more evident, e.g. costly. One expects to have a major system design 
effort and system integration testing. In reality the tasks that 
constitute such activities are almost always performed (although 
usually subsumed in the planning) on even the smallest computer pro- 
gramming job that results in an operational computer program. 

The steps in the programming process imply a sequence in which the completion 
of work within each step is prerequisite to the start of work on the following 
step. In practice, however, this principle may not be apparent, because a 
project may be divided into subprojects wherein work proceeds at different 
rates for various subprojects. Also, work completed on a process step may 
have to be repeated because of later developments in the process; for example, 
the failure of a program to pass an integration test will require repeating 
some part of the program design, code, and test step, or the system analysis 
and design step, or both. This recycling within the programming process, 
illustrated in Figure 2, is widely recognized as a major factor in both the 
magnitude of total costs and the lancertainty of predictions for programming 
costs. 

In conducting the analysis that led to the results in the section on Computer 
Program Design, Code, and Test, we assumed that computer programming, regard- 
less of application and the types of resources used, has certain characteristics 
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that can be generalized^ and that variation in costs from job to job can be 
accounted for by the variation in a selected set of these characteristics^ or 
cost factors. Thus^ the primary costs ^ manpower and machine usage ^ can be 
considered as dependent variables that can be expressed as a function of these 
cost factors (independent variables). An implicit assumption is that the pro- 
grams whose cost data form the data base are representative of those of any 
user of the Handbook. This^ of course^ will not usually be true. 

To try to tailor the analysis so that more useful results could be derived for 
specific programming organizations^ we divided the data into classes to dis- 
tinguish a number of different kinds of computer programs^ such as business^ 
scientific^ and programs written for large computers. Even more work of this 
type is desirable. However^ the size of our data base does not permit 
statistical analysis of finer divisions that might be useful for characterizing 
the computer programming work in a particular organization. 

6. Interpretation . All the data used from both the statistical analysis and 
the literature were data of opportunity^ i.e.^ we took what we were able to get 
in the time available. Hard data on the costs of computer programming or^ more 
generally^ automatic data processing economics are scarce commodities both in 
computer programming organizations and in the published literature. Few 
numerical data are recorded; fewer yet are recorded under "controlled" 
conditions^ and still fewer are suitable for generalization to other situations. 
In the rapidly multiplying field of computer programming^ diverse opinions on 
the costs and benefits of techniques and applications are rampant. This wide 
variation in the literature is matched by the variation in the numerical data 
that we used in the statistical analysis. The respondents to the questionnaire 
were under no obligation to assure completeness and accuracy even when data 
were readily available. Because they were suspect^ some of the data collected 
were rejected prior to the analysis. But even those data used in the analysis 
are likely to have a variation in reliability^ which we believe has been 
smoothed by the use of statistical techniques applied to a sample of reasonable 
size. 

The user of this Handbook is likely to discover conflicting data fran other 
sources^ and obtain different estimates using the various material supplied 
by the Handbook. These conflicts characterize the state of the art in cost 
estimation and indicate the limitations of the data base available to this 
project. Specific limitations of the data base used in the preparation of the 
Handbook include inaccuracies in reporting data (including guesswork by 
respondents)^ misinterpretations of questions by respondents^ inclusion of 
costs that were not a part of the program design^ code^ and test step^ and 
inappropriate definition of data items or cost factors on the questionnaire. 
We assume that the reader will recognize the limitations of the data in 
applying them to his specific problems. 
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In this sense, the Handbook can be considered a first step to supplement (not 
to replace) judgment by managers. Because the predominant trend in planning 
for computer programming has been to underestimate the costs, the best advice 
that can be offered at this time is to lean toward the conservative (realistic) 
estimates for costs. 

The following breakdown lists the number of different types of computer programs 
supplied by each source that contributed to the data base. This list may be of 
some assistance to the reader in making inferences about the applicability of 
the Hajidbook data to his specific application. 
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SECTION II 
USING THE HANDBOOK 

To explain the use of the Handbook, this section describes (l) the structure of 
the remaining sections (corresponding to the steps in the programming process), 
including the formats for the material, and (2) a vay in which the estimates 
that result from use of the guidelines can be introduced into the planning and 
an evaluation of costs for a proposed effort in computer programming. We con- 
clude the section with some brief advice on how to supplement the Handbook 
based upon our experience in gathering and analyzing data. 

1. Structure of the Handbook . Exclusive of this section and the Introduction, 
the Handbook is divided into six basic sections — one for each step of the 
programming process for which costing infomiation is supplied. These data for 
each activity in the programming process are presented in a uniform manner. 
Each section contains the following kinds of formatted material: 

. An activity definition that includes a list of major tasks, inputs, and 
outputs for the particular programming step. 

. Major cost factors—a list, along with some indication of the impact of 
these factors, followed by discussion of those factors for which we 
have additional information. 

. Planning factors that can be used with unit price and percent-of-other- 
item estimating methods described in the Introduction. 

. Estimating equations currently available only for the Program Design, 
Code, and Test activity). 

In any section, a symbol, a replica of Figure 1, appears at the top of the 
first form of each of four types and indicates, by shading, which activity or 
step is being addressed. The forms and the types of information these contain 
are described in more detail below. 

a. Activity Definition . The Activity Definition for each step in the 
computer programming process is presented as shown in Figure 3 with the 
following items : 

(l) Description . The description is a concise general statement of 
what the activity is. In addition, for readers that are involved in electronic 
system development for the Air Force, we have included information that 
relates the sequence of activities to the planning and control for computer 
programming that is reflected in the Air Force guidelines on Systems Management 
that will be found in the Air Force Systems Command Manuals in the 375 series 
(6, 61). 
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ACTIVITY DEFINITION 



ACTIVITf; INTORMATIOU PRCCESSIKG SYSTEM INTEGRATION TEST 

DESCRIPTIOH: This activity covers all work necessary to test the 
performance of the computer program within ths total 
system at the operational facility under realistic 
("live") operating condition*. 

(AFSCM 375 context: This activity occurs in the 
Acquisition Phase and is equivalent to Category II 
testing. ) 

TASKS: Conduct Test . Within the requirements of the plans for 
program testing, conduct a sequence of tests of the 
total operational system, receiving actual data from 
and transmitting actual data to other subsystems and 
components that constitute the system. 

Analysis of Test Results . Determine if the system 
meets specifications, and study operations for any 
evidence of potsntial difficultisa. Coordinate with 
other subsystem and component developers to isolate 
sources of poor system perfomance. 

Initiate Modifications to Cowputer Programs . Correct 
errors in programs and associated docisnentation. Design 
and implement feasible changes to meet specified 
perfomance requirements. This involves vork in the 
earlier activities. 

Documentation of Test Results . Prepare appropriate 
compliance documentation to certify successful tests. 
If problem areas arise, document the evidence for use 
in the modification process. Identify potential 
improvements that could be made to the total system. 



FIGURE 3 
SAMPLE ACTIVITY DEFINITION FORMAT 

(2) Tasks. To detail each activity, a checklist of the major tasks 
that may be part of the activity is provided. Not all computer programming 
efforts vill involve all of the tasks listed. 

(3) Inputs and Outputs . The inputs and outputs describe the informa- 
tion required to perform work on the specified activity, and the products of 
the specified activity, respectively. 

The documents such as specifications and statement listings, cited as outputs 
of each activity, are the direct products of the programming process that are 
needed to perfonn the next step or to aid in the operation of the computer 
program by the user. We excluded items prepared for technical management such 
as activity reports, and project control and cost reports. Each output listed 
for a process step provides not only a basis for understanding the activity, 
but also the basis for control of the programming project by using a deliverable 
item with a completion date as a milestone. 
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FIGURE k 
SAJ^1H.E CO^^IRJTER PROGRAMMING COST FACTORS FORl^T 



b. Ma.ior Cost Factors. 
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The next type of form found in each section (see 

Computer Programming Cost Factors") lists cost factors, i.e., those 



Figure k, 

variables that affect the expenditure of resources in the particular activity 

being addressed. Four types of resources are used in computer programming: 

(1) Manpower measured in units such as man months. 

(2) Computer usage, measured in units of time such as hours of use for 
ma in- frame of a computer. 

(3) Elapsed time, measured in units of time such as months. 

(k) Other ADP costs, such as expenditures for supplies expressed in 
dollars . 

The canpleted form, as in Figure ^, shows how each factor influences the first 
three resources--manpower, computer usage, and elapsed time. Other costs, 
such as those for supplies, are not discussed in this Handbook. The form in 
Figure k contains seven columns. The first column contains the names and 
numbers of cost factors that are defined more completely in Glossary A. The 
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next three columns^ labeled "Impact on Indicated Resources," contain informa- 
tion on the influence of the factor upon the three major costs— manpower, 
computer usage, and elapsed time. 

In the section on the Computer Program Design, Code, and Test activity, the 
entries in these three columns contain a sign (plus or minus) indicating 
whether or not a factor increases or decreases costs, and a number (based 
upon a statistic—the correlation coefficient) that indicates the amount of 
the impact. In the other sections for the other five activities, since no 
statistical analysis vas conducted, only the direction (sign) of the influence 
is shown in these three columns. 

The next three columns in Figure h are labeled "Reference." The first column, 
"Handbook Page," contains numbers of pages in the Handbook where additional 
information on a particular factor can be found, e.g., in the discussions 
immediately following the list or in the planning factors. The next two 
columns, labeled "Descriptive" and "Analytical," contain the numbers of refer- 
ences that discuss the particular cost factor. Descriptive references do not 
contain numerical data, while Analytical references do. 

These lists of factors (and the discussions that follow them) provide at least 
two benefits to the user of this Handbook. First, the estimator can interpret 
and evaluate his own situation vis-a-vis the planning factors and the 
estimation equations presented later. That is, he can consider the relative 
impact of the various cost factors on his specific problem to help him decide 
on which estimating relationship to use. Secondly, the list of factors that 
affect cost can be used as a checklist for setting up a system to control costs 
by managerial attention to the causal relationships involved. For example, the 
importance of stability of design as a factor in total costs argues for 
thorough initial planning and a management policy that minimizes changes in 
system requirements. 

The cost factors are not necessarily independent, although this would be 
desirable. On the contrary, the statistical analysis on the numerical data for 
Computer Program Design, Code, and Test showed that many of the factors are 
highly intercorrelated (e.g., X^s—machine access time and X59— machine add 
time, X3_o~total object instructions and Xi^^'-'J^umber of conditional branches). 
There may also be considerable difference of 'opinion as to precisely how a 
factor should be defined or measured. But the purpose of this Handbook is not 
to resolve these matters; instead, it is designed to provide a useful list of 
factors and definitions, and a format such that the list of factors and 
definitions can be added to or modified to meet the particular needs of 
different users. 

c. Planning Factors . The third type of information found in the remaining 
sections of the Handbook is labeled "Planning Factors." The data on forms 
permit the user to use the unit price and per cent-of-other- item methods for 
estimating costs. These methods require that the user have information on the 
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number of items needed or information on the cost of the other items. Even 
when one or the other of these two pieces of information is available^ the 
user must still use judgment to decide which planning factor to use because 
there may be several that could apply. 

As indicated earlier^ most of the numerical data in the Handbook appear in the 
section on Computer Program Design^ Code, and Test, An example of a completed 
Planning Factor form that appears in this section is shown in Figure 5. This 
example presents unit price data — man months, computer hours, and months elapsed 
per 1000 object instructions for various subsamples such as language type, 
application type, and computer type. 

This use of number of instructions as a normalizing parameter, i.e., instruction 
as the unit in the unit-price data, is common throughout the Handbook particu- 
larly in the section on Computer Program Design, Code, and Test, In using this 
device we do not distinguish among different computers in terms of logical 
power of the instructions. Also in the section on Computer Program Design, 
Code, and Test we do not account for the characteristic inefficiency of 
Procedure-Oriented Languages and their associated compilers. Because of this 
inefficiency, the use of a POL yields, on the average, more machine-language 
instructions than use of Machine-Oriented Language (symbolic or assembly 
language) and an assembler, to perform the same logic or solve the same problem. 
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FIGURE 5 
SAMPLE PLANNING FACTORS FORMAT 
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In addition to the form shown in Figirre 5, other forms will be used to present 
the planning factors^ e.g., plots of data to exhibit the percent-of-other-item 
cost data. Also, in some instances where data are available, planning factors 
for tasks or subtasks that comprise one of the major activities in the computer 
programming process will be listed. 

d. Estimating Equations . As indicated earlier, estimating equations are 
only available for the Ccxnputer Program Design, Code and Test activity. These 
results of the current analysis conducted by the Programming Management Project 
include equations for estimating manpower, computer usage, and months elapsed 
that are derived from and apply to the entire sample as well as various sub- 
samples. The results of the subsample analyses were included only if the 
equations derived were more accurate (i.e., had some smaller errors) than the 
equations for the entire sample. The format in which the equations are 
presented, for example, as in Figure 6, contains the following: 

(1) The equation itself in terms of numbered variables, X.. The 
variables are defined on the fold-out (page 89) • 

(2) In addition to the identification by table number, a heading that 
describes the sample under Data Base and the resources being estimated under 
Resource. 



DATABASE: 

TOTAL SAMPLE N - 169 



MAN-MONTHS 



TOTAL MAN-MONTHS ^Y, 



.40 X^, 
58.82 X^ 



42 
30.61 X^, 




MANMOMtHI 

HISTOGRAM Of MAN MONTHS 




FIGURE 6 
SAMPLE ESTIMATING EQUATIONS FORMAT 
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(3) A frequency distribution in the form of a histogram for the data 
that were collected for the indicated resource. 

(h) A plot of estimated values using the indicated equation on the 
factors for each data point against its actual values for the particular 
resource in the data base. This plot also contains Stanine Bands to portray 
the statistical confidence in the derived estimating equation. 

On plots such as the Estimated versus Actual Man Months in Figure 6, the confi- 
dence levels are indicated by the inserted table, Stanine Band Number versus 
Probability Values. These bands can be interpreted as follows (l6): Suppose 
an estimate of 80 man months has been calculated. By reading on the vertical 
line for this value in Figure 6, we see that the probability (or chance) that 
the actual value will fall between 68 and 92 man months (the values of 
boundaries on the Number 5 Stanine Band) is found in the table as .20 (or 20 
out of 100 programming jobs). Using the same estimate of 80 man months, the 
probability that the actual value will fall between 53 and 107 (the lower and 
upper boundaries for the Number h and Number 6 Stanine Bands respectively that 
bracket the Number 5 Stanine Band) is .^h, the sum of the probabilities shown 
in the table for the Numbers h, 5^ arid 6 Stanine Bands. Note that the bands 
are symmetric around a if5-degree line and their width is equal to one-half the 
standard error of estimate given in the upper left-hand corner of the plot. 
The standard error depends upon the sample size and the power and efficiency of 
the predictors used in deriving the equation. Another sample may widen or 
narrow the Stanine Bands leading to changes in the estimating precision. 

2. The Preliminary Cost Evaluation Analysis > The Preliminary Planning and 
Cost Evaluation activity, the first step in the computer programming process, 
includes the actual estimation of costs. Section III in the Handbook describes 
the activity and planning factors for estimating the resource cost of the step 
itself. This step is unique; it not only triggers the work on the other steps 
of the programming process, but interfaces with other management activities. 
The output of the cost evaluation step may well determine whether or not the 
project should be continued. 

To show the relationship of cost estimation to planning or, in particular, how 
the cost estimates made by using the material in this Handbook can be used in 
a cost evaluation for a proposed computer programming effort as part of the 
development of a data processing system, we discuss the Preliminary Planning 
and Cost Evalixation work in terras of four forms; 

Figure 7; A Sample Cost Justification Form that provides a way to 
compare costs for a proposed computer program as part of on ADP 
system with the costs for an existing system (automatic or manual) 
that performs the same data processing functions. 
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Figure 8, A Sample Project Description Form that would contain the 
numerical values for cost factors such as system and resource charac- 
teristics to describe the proposed computer programming project. 
These factors are the inputs needed to use the cost-estimating 
relationships in the Handbook. 

Figure 9; A Sample Budget/Schedule Work Sheet that could be used to 
record the allocation of resources (both in resource xinits, e.g., man 
months and dollars) for each activity over several intervals of time 
as v;ell as to record a total for each activity (summed over the time 
interval) and a grand total for the entire project. 

. Figure 10, An ADP Project Budget/Schedule Summary that can be used to 
record the costs from Figure 9 by fiscal year intervals and to combine 
the computer programming costs with equipment costs. To relate it to 
Air Force and DOD planning, the form provides for division of the 
costs into Research and Development (R and d), Investment, and 
Operating costs, the categories used by the Air Force in preparing 
a long-term plan (^2). A provision for discounting costs is also 
made in the form. 

These four forms (Figures 7 through 10) should be viewed as suggestions for 
ways to record the evaluation of computer programming costs and to integrate 
these costs into long-range plans. 

a. The Cost-Justification Format . The cost-justification procedure 
outlined here consists of a comparison of the total costs (development and 
operating) expected for the proposed system (or project) with the operating 
costs of the present system. This comparison can be conveniently made on a 
form such as illustrated in Figure 7, a data processing project cost evaluation 
summary. This form is aimed at comparing costs for an ADP system or project 
that requires computer programming work as part of the development and 
operations. The right-hand side of Figure 7 contains summary costs for the 
proposed system, the left-hand side, the summary costs of the existing system. 
Modifications of this form might include provision for several alternatives 
in design for the proposed system. The meaning of each item of Figure 7 is 
as follows : 

Item 1, Assumed Life of the Project. Operating costs accrue continuously, 
or at various time intervals, for the life of the project. If the value is 
to exceed the cost, ultimately, the nonrecurring costs of procuring a system 
must be amortized by savings in future operating costs. The total estimated 
life of the project (measured in years) during which these savings may be 
realized is recorded in Item 1. 
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Item 2, Proposed Project Nonrecurring Costs , Nonrecurring costs are those 
associated with the procurement and installation of a new system. This 
Handbook was specifically intended to help in making estimates for Items 2a 
through 2e in Figure 7; that is, the costs of the various activities in the 
computer programming sequence. The costs of purchase (item 2f) and 
installation (item 2g) of any new equipment are also identified as non- 
recurring costs for the project under consideration. We also include in 
this category one-time costs such as the costs of personnel training, 
purchase and installation of non-ADP equipment or material or project 
costs such as management procedures. 

Item 3, Present System Operating Costs . These costs (items 3a through 3^) 
for an ADP system include hardware maintenance, rental and operation, as 
well as program maintenance. If the present data processing system is 
entirely manual, these annual costs would be summarized under the "Non-ADP 
Costs" heading. If the proposed data processing system is a completely new 
function, with no existing system to replace, the present system costs would 
be zero. In this case, economic justification would be based entirely on 
the proposed system costs and Item 9^ in Figure 7> added benefits of the 
proposed system measured in dollars. Since investment, that is, the non- 
recurring, costs for the present system are sunk costs, they are not 
considered (k) . That is, the funds have been committed and are not 
eligible for euLlocation in making this particular decision. 

Item k, Proposed Project Operating Costs . The types of items in annual 
operating costs for a proposed ADP system are the same as for the present 
system (item 3) and include computer hardware maintenance (item ^a), computer 
rental (item ^b) or payments if equipment is not to be purchased outright 
(in the latter case for purchase cost, the down payment would be included in 
Item 2f), and ADP operation costs; ADP operation costs (item ^c) include the 
costs of operating personnel and power. Annual maintenance programming costs 
(item hi) are those associated with this function as defined in Section VIII 
of this Handbook. Other annual ADP operating costs (item ^e) include supplies, 
keypunch, and all other recurring ADP related work. 

Non-ADP operating costs cover the annueuL costs of associated manual systems, 
the proposed project's share of overhead, direct supervision, and all other 
recurring costs not previously accounted for. 

Item 3} Total Present System Costs . This item in Figure 7 represents the 
sum of all the annual costs in Item 3; multiplied by assiiraed life (in years) 
of the proposed system. 

Item 6, Total Proposed System Costs . This item in Figure 7 represents the 
sum of all annual costs in Item h multiplied by the assumed life (in years) 
of the proposed system, plus the sum of all nonrecurring costs in Item 2. 
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Item 7; Discounted Total Cost of Present System ^ The present value of the 
current system cost (Item 5) discounted at the indicated annual interest rate 
("the value of money") over the time period for the assumed life of the system 
is summed Etnd entered here. If discounting is not desired (i.e., if the 
rate = O), this value is the same as entered in Item 5» 

Item 8, Discounted Total Cost of Proposed System . The present value of the 
proposed system cost (Item 6), discounted at the indicated interest rate over 
the time period for the assumed life cycle, is summed and entered here. The 
interest rate Eind time period for discounting proposed system costs to their 
present value must, of course, be the same as used for the present system. » 
If discounting is not desired, this value is the same as entered in Item 6. 

Item Sj Justification . The xiltiraate justification of a proposed ADP project 
depends on the expected savings and/or additional dollar benefits expected. 
This calculation may be readily made in Item 9 of Figure 7. Note that 
expected savings (i.e., cost of old system less cost of new system) may be 
negative, and indeed they must be when the proposal is not a replacement 
for an existing procedure; the entire burden of acceptance then rests with 
the expected ability of the proposed system to generate new dollar returns. 
The procedure for calculation of expected economic returns for proposed ADP 
projects is a subject beyond the scope of this Handbook. 

The final decision to proceed with a project may depend upon many more 
considerations than merely obtaining a positive number for the total economic 
value in Item 9c of Figure 7. For example, the absolute magnitude of the 
proposed system cost (item 6) may exceed available resources. Another 
important consideration is the percent of expected total value (item 9c) 
to total proposed system cost (item 6); this is because of the uncertainty 
attached to any estimate, and the possibility of higher-than-expected costs 
wiping out anticipated returns. 

b. Defining System Characteristics . The cost-evaluation summary illustra- 
tion in Figure 7 provides an overview of the kinds of estimates required to 
justify the cost of an ADP project. To develop actual numbers for the costs 
of a proposed project, the cost factor values for the project being estimated 
must be specified. Figure 8 provides a siiggested format for this. 

Items 1 through 9 and 2U through 26 of Figure 8 are self-explanatory. Items 10 
through 23 should contain the names and the numerical values for the important 
cost factors (system and resource characteristics) that are known or can be 
estimated at this early stage. To determine what is important, the user should 



For a discussion of discounting in return on investment analysis, see (^). 
The time phasing of expenditures may be worked out on forms such as Figure 9. 
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PROGRAMMING PROJECT DESCRIPTION 


1. PROJECT TITLE: 


2. EXPECTED 
START DATE: 
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COMPLETE DATE: 
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7. PROJECT MONITOR: 
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11. 
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14. 
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21. 


22. 


23. 


24. PREPARED BY: 


25. ORGANIZATION: 


26. CURRENT DATE: 
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read through the other sections in the Handbook as veil as Glossary A, the 
Definitions of Cost Factors, and use his judgment. For example, these entries 
could include the independent variables such as those used in the estimating 
equation for Program Design, Code, and Test or the basis for the selection 
of particular planning factors from the range of values available in the 
Handbook . 

The Sample Project Description Form, Figure 8, is one way that such information 
may be summarized and presented. Other formats and data items may be desired, 
or even contractually required, in certain instances (2). For example, sub- 
ordinate forms could be prepared that describe the work to perform each of 
the various steps. These forms could list factors that influence the cost 
of the particular step as veil as their estimated values. 

c. Making a NumericeuL Estimate . To prepare the numerical estimate for 
use in the Cost Evaluation Summeo^y, Figure 7^ information such as that recorded 
in Figure 8, Programming Project Description, is used with the guidelines in 
the Handbook in the following manner. Figure 9^ The Programming Project 
Budget/Schedule Work Sheet, is a suggested form in which only the programming 
costs could be entered. 

(1) Select the appropriate planning factor or estimating equation, 
for each step in the computer programming process. This selection is guided 
by the system characteristics and project description (as in Figure 8) deter- 
mined in the previous step. The selection may also be influenced by the desire 
of the estimator to provide a margin of safety to cover the many uncertainties 
involved. When statistical measures are available that show the expected 
spread of the estimated costs, such as standard deviations for planning 
factors or Stanine bands for equations, these yield important insights into 
the statistical uncertainties inherent in the data base. 

(2) Calculate total man months, computer hours, and the other 
resources (e.g., supplies) required for each separate step in the programming 
process, using the planning factors and/or equations selected above. In 
Figure 9^ these entries would be made in column 11. 

(3) Distribute the estimated totals over an appropriate time span. 
The work sheet. Figure 9^ divided into periods appropriate for the size and 
time span expected for the project, provides for recording these estimates. 
This preparation of the plan for performance is guided by the following 
considerations : 

(a) The normal sequence of steps in the computer programming 
process. If the computer program for the project is divided into subprograms 
with different schedules, additional work sheets similar to Figure 9 would be 
desirable for scheduling and budgeting each subprogram. Of course, the 
schedules for all subprograms should intersect at the end of the Computer 
Program Design, Code, and Test activity when the total program system is 
tested as a unit. 
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(b) Any constraint on the total time available for completion. 
Delivery dates, dictated by external reqixirements, may determine the total 
number of personnel working on the project from the total man months required 
and the delivery dates specified, 

(c) Any constraint on the resources such as manpower or computer 
hours available per time period for the project, 

(d) Preferred time spans for the activities based on the technical 
factors involved. The Planning Factors and Estimating Equations in this 
Handbook will help the user estimate the amount of elapsed time needed in 
various situations, 

(k) Convert the resource units (man months, computer hours) into 
dollar values. That is, the resource units are multiplied by the dollar cost 
per unit applicable at the facility in question. Figure $, the Budget/Sched\ile 
V/ork Sheet, provides for entries that resiilt from these calculations, 

(5) Check for reasonableness of estimate. For example, make a 
direct comparison of the costs of the project in question with the historical 
costs of other similar projects. This technique is a variation of the specific 
analogy method of cost estimating described earlier. In certain circimistances, 
it may also be desirable to obtain the advice of experts or consultants, e,g., 
by securing an outside bid on portions of the work (39)^ 

(6) Integrate the dollar costs estimated for computer programming 
such as those recorded in Figure S, the Budget/Schediile Work Sheet, into an 
overall budget for the ADP project that includes equipment and other costs. 
Figure 10 is an example of a form in which such information may be Siunmarized; 
data for the items that are steps in the programming process are derived from 
Figure 9^ Th^ items in Figure 10 are divided into Research and Development 
(R&D), Investment, and Operating categories; this division corresponds to the 
format used by USAF in long-range planning. Figure 10 also provides for 
entering the results of a discounting calc\ilation, 

(T) Incorporate dollar estimates into the cost-evaluation smmnary 
such as Figure 7. 

This estimating process wi3_l usually be repeated through several iterations 
during the progress of a programming project (for example, see Figure 2). 
During the first step of the programming process described in Section III of 
the Handbook, the first preliminary estimate is made. If the decision is 
made to proceed, the information system analysis and design step will usually 
provide revisions to some of the original estimating assumptions, thus requir- 
ing a new cost estimate. Also, changes in the information system design and/ 
or specifications may be made at various stages in the programming process, 
at the instigation of management or the customer, or because of the results 
of testing; such changes may make new cost estimates advisable. And finally, 
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the management control system for the programming project may indicate over- 
runs and/or slipping schedules, requiring a revision of estimates or a 
modification of objectives or both. 

3. Supplementing the Handbook * As indicated earlier, one purpose of the 
Handbook is to encourage managers to adopt an analytical viewpoint and to 
collect and analyze data on their own computer prograimning operations. 
Specific suggestions on the types of data that ceui be collected caji be 
found in the Computer Programming Cost Factor forms in Sections III through 
VIII. Glossary A defines these variables in more detail. Techniques ajid 
procedures for ajialyzing the data are described in earlier reports by the 
Programming Management Project at SDC(l9^ 20, 62) • These documents contain 
references that provide even more details on the statistical techniques. 

Analyses of local operations have certain potential advajitages and disadvan- 
tages. These pros ajid cons are discussed below along with some recommendations: 

a. Advantages 

A majiager responsible for a single programming organization caji 
greatly reduce the number of variables (cost factors) considered 
in this Handbook by selecting or defining those that he feels 
apply to his particular situation, e.g., type of job, equipment, 
personnel resources. This reduction, in turn, simplifies and 
hence reduces the cost of the ajialyses of the data. 

A manager caji more readily identify those cost factors susceptible 
to control in his own operation ajid can actuetlly control them. As 
a result, he can reduce the variation in costs ajid improve his 
planning. 

Also, he can identify and measure the impact of the cost factors 
that cannot be controlled (usually variables that characterize 
requirements) and thereby improve his pricing techniques. 

With an accumulation of some data that provide local standards, a 
majiager can more easily identify and account for the cost differ- 
ences associated with changes in programming techniques, equipment, 
personnel, personnel training, orgajiization, and technical 
management policies. 

b. Disadvajitages 

On the other hand, local data collection and ajialysis may not be 
effective because sufficient data cannot be collected within one 
organization to derive local standards for cost that will be useful 
for control and planning. Although the number of factors can be 
reduced, thus permitting a smaller sample to be used in the analysis, 
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too drastic a reduction could purge factors that actually account 
for variation in local costs and the analysis nay yield poor 
estimating relationships. 

Using more factors as a basis for data collection requires a larger 
sample (more data points corresponding to computer programming 
efforts) to derive equations. To accumulate data on an adequate 
sample may take a long time in one organization because a single 
prograjnming effort may take several months to complete. 

Time and money are needed to collect and analyze data. Further ; 
some discipline is required to "reserve" the time of the personnel 
responsible to assure that the collected data are actually analyzed. 

c. Recommendations . A compromise might be to have uniform data collected 
locally and forwarded to some central organization where these data would be 
analyzed. Useful resiilts would then be fed back to contributing organizations. 
The cost analysis that led to the results in Section V, Computer Program Design, 
Code, and Test, was an approximation to this approach. However, these data on 
completed programming projects were not usually based upon records made while 
the projects were under way, and some of the data were thus unreliable. 
Another problem in data accuracy stemmed from the absence of face-to-face 
interviews in collecting data; terms used in the mail-out questionnaire were 
misinterpreted. As a result, additional time was used to check and correct 
the data, and even then, some data were discarded. 

To anticipate such problems and promote the useful collection of data, we 
recommend the following: 

Despite the disadvantages, do collect and analyze data. Even if 
strong statistics are not realized quickly, insight into the 
distribution of costs will be gained. 

Collect the data in terms of computer programming products, i.e., 
deliverable end items. The usual question on pricing and budgets 
is, "What will it cost to do a particular job?" 

Define the costs and cost factors as precisely as possible in 
terms that are common to other organizations. This permits the 
manager to use (l) data and cost-estimating relationships from 
other prograjnming organizations or cost studies, and (2) any cost 
standards that may be developed by Federal Government agencies. 

Collect data while the projects are under way, by activities (as 
in this Handbook) or by selected milestones. If the data are not 
recorded as they are generated, personnel must rely on recall or 
search through related records, reducing the reliability of the 
data and adding to the cost. 
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Identify a central repository and person who is responsible for 
the storing of the data (an organization with many programming Jobs 
coiild use automatic data processing techniques) to avoid loss of 
parts of the data. 

Record initial estimates of the costs for which standards or 
estimating relationships are desired, and feedback from analyses 
that include comparison of early estimates with actuals. This 
will close the loop and provide the most immediate benefit to the 
responsible personnel in making improved estimates. Also, these 
personnel may be able to identify reasons for differences, between 
actuals and estimates; these can be used to create new cost factors. 

Identify and record reasons for revisions to estimates during a 
project. Ageiin, these are potentisuL cost factors. 

Do not start a data collection, unless the intent and resources 
are to be made available for analyses and feedback. If the feed- 
back loop is not completed, costs expended are largely wasted and 
personnel may be left with impressions that would cause them to 
resist future data collection efforts. 

Test any proposed scheme for data collection on a small sample of 
projects to assure that the scheme is feasible and \inderstandable 
by the range of personnel who will have to provide source data. 

Try to forecast the changing technology in any data collection 
effort, and be prepared to make changes to accommodate any 
unanticipated new developments. 

Finally, try to keep some data on the cost of the data collection 
and analysis work itself, and compare these costs with the benefits 
derived. 
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TABLE L 



ACTIVITY DEFINITION 



ACTIVITY: 
DESCRIPTION: 



TASKS: 



PRELIMINARy PLANNING AND COST EVALUATION 

This activity consists of the economic feasibility 
study for the proposed program. Based on a statement 
of the user's requirements, an estimate is made of the 
manpower, computer time, elapsed time, or other resources 
required for the project. Using these estimates, a 
siammary project plan and a cost versus benefits com- 
parison are prepared. No more emeilysis of the proposed 
information system is done diiring this activity than 
is absolutely necessary for cost estimation and 
preliminary planning ijurposes. 

Determine System Requirements and Characteristics . 
Study user's requironents. Make a preliminary analys is 
of the environment (organization, heirdvare, data base 
requirements), emd any similar existing computer 
programs to determine the veilues of the parameters 
necessary for a cost estimation. 

Select Appropriate Plemning Factors . Based on the 
cheiracteristics of the proposed system, select the 
appropriate planning factors and/or cost estimating 
equations to be used. Soiirces for these factors 
include this Handbook, other published materieil 
emd historical records of the installation making 
the estimate. 

Ceilculate Computer Programming Costs . For each step 
in the programming process, calculate the man months, 
computer ho\irs, and other resources required for the 
proposed project using the previously determined 
planning factors. Convert units to dollar equivalents. 
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TABLE. 



ACTIVITY DEFINITION (CONT'D.) 



Determine Project Costs Other than Programming , Prepare 
estimates of other costs associated with the proposed 
project. These costs include: equipment purchase 
and/or installation; equipment maintenance, rental, 
and operation; data base conversion; overhead costs; 
personnel training; supplies. 



Check Reasonableness of Estimates. 



Compare estimates 
Compare totals with 



PRINCIPAL 

INPUTS: 



made by several different methods 

those of similar projects. Obtain expert opinion. 

Prepare Summary Budget Plan . Develop project schedules 
by allocating cost expenditures over time. Prepare 
si;unmary (i.e., by step in the programming process 
within major subproject breakdown structure), Gantt 
and/or PERT charts, and budgets for performing 
organizations. 

Determine Costs of Existing System . Determine the 
costs of accomplishing the objectives of the proposed 
project with the current system. Establish a dollar 
vEilue for gains of the proposed project that are over 
and above the benefits expected from replacing a 
present system. 

Determine Economics Feasibility of Proposed System . 
Compare the costs and benefits of the proposed system 
to those of the existing system. Determine if the 
absolute magnitude of the costs involved is within the 
limits of resources of the organization. Determine if 
the required resources, such as programmers, will be 
available within the established schedules. 

User's requirements and environment. 

System requirements and environment. 

Planning factors and cost-estimating relationships. 

Unit dollar costs of resources for the facilities 
involved. 

Total resources available to the project at the 
facilities involved. 
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TARi F I ACTIVITY DEFINITION (CONT'D.) 



PRINCIPAL Project description 
OUTPUTS: 



Resource-iinit, e.g., man months and dollar-cost 
estimates. 

Tentative schedules. 

Cost justification project evaluation summary. 
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TARI F II . COMPUTER PROGRAMMING COST FACTORS 


FACTOR 


♦IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


HANDBOOK 
PAGE 


OTHER 


DESCRIPTIVE 


ANALYTICAL 


Requirements 














X. - Vagueness of design 

requirements definition 


+ 




+ 








X- - Lack of knowledge of operational 
requirements 


+ 




+ 




















X_Q - Complexity of system interfaces 


+ 




+ 








XoL - N\3mher of functions in system 


+ 




+ 
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Program Design 8c Production 














X,- - Total source instructions 
■^ expected (i.e., expected size 
of program) 


+ 




+ 


38 


















X ^ - Number of subprograms 


+ 




+ 
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Xi J- , - Internal documentation expected 


+ 




+ 








X|^ - External documentation required 


+ 




+ 








X. - ^ - Total number of external 
' * document types expected 


+ 




+ 




















X. „ - - Total number of internal 
'*^ document types expected 


+ 




+ 




















Xkq - Type of program 














*NOTE: IMPACT IS INDICATED BY: 




+ = VARIES DIRECTLY 




- = VARIES INVERSELY 




NO SIGN = NO PRESUMED DIRECTION 




(FOR DISCUSSION OF CODING, SEE GLOSSARY D) 
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TABLE JEL. 



COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 



FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 



* IMPACT ON 
INDICATED RESOURCE 



MAN 
MONTHS 



COMP. 
HOURS 



ELAPSED 
TIME 



REFERENCE 



HANDBOOK 
PAGE 



OTHER 



DESCRIPTIVE ANALYTICAL 



Data Processing Equipment 



^51 
S3 



First program on computer 

ADP components developed 
concurrently 



X/rn - Number of agencies concurring 
in design 

Development Environment 



'69 



^81 



'88 



'89 
^90 



- Customer inexperience 

- Number of sources of system 
information 

- Accessibility of system 
information 

- Quality of resource documents 

- Availability of special tools 

- Degree of standardization in 
policy and procedure 
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ADDITIONAL COMMENTS ON SELECTED COST FACTORS 



13 



Total Source Instructions Written * The rationale for using total source 
Instructions vrltten (or also, perhaps, X,q, Total Object Instructions 
Delivered, "^^j? Number of Conditional Branches) as a cost factor for 
estinating tne cost of the planning Is that this variable Is a principal 
measure of program size. The hypothesis Is that larger sized programs 
require more thought and effort to make an estimate of their costs. 
Most of the time for a nev ADP system, an estimate of this cost factor 
will be difficult to obtain. 



^Ul 



^8U 



'89 



Number of Subprograms * Division of a large computer program into sub- 
programs may be a natural" classification of data processing tasks, 
may help achieve a more modular design for ease of maintenance and also 
may facilitate the division of labor In computer programming. If sepa- 
rate groups of personnel are to develop these subprograms, then separate 
cost breakdowns should be made for these subprograms. Therefore, the 
number of these subprograms would substantially affect the cost of 
making the total estimate. 

Number of Functions in the System * The number of functions in the system 
would affect the cost of making an estimate in at least two ways. First, 
the larger the number of functions, the more factors to be considered In 
attempting to determine the magnitude of the programming job. Second, 
the larger the number of functions affected, the greater the task of 
determining the present costs for the cost comparison (Figure 7). 

Availability of Special Tools * For the cost estimation step, special 
tools to aid In the estination process would Include worksheets and 
planning factors such as those found in this Handbook. Also computer 
programs, based on equations and planning factors found in the Handbook, 
could be prepared to automatically do portions of the cost estimating 
task. 
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TABLE III 



PLANNING FACTORS 



TYPE: 



VARIOUS 



SUBJECT: 



ESTIMATION OF TASKS WITHIN ACTIVITY 



TASK 



ESTIMATION RULE 



Establish System 
Characteristics 



Select Planning 
Factors 



Up to one man day per subprogram. 



Calculate Programming 
Costs 



Determine Project Costs 
Other than Programming 



Check Reasonableness 
of Estimates 



Prepare Sxmimary 
Budget Plan 



Determine Costs of 
Existing System 



Determine Economic 
Feasibility of 
Proposed System 



Allow one man day for each expert 
consulted. Allow vcp to one man day 
per subprogram for each alternative 
estimate. 



Allow one man day. 



Costs of this step vary from those of 
an off-hand estimate to an elaborate 
study. For some concepts and methods, 
see (28). 



Allow 1-2 man days for summarizing 
data and briefly docimienting 
recommendations . 
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TABLE, 



IV 



ACTIVITY DEFINITION 



ACTIVITY: 



INFORMATION PROCESSING SYSTEM ANALYSIS AND DESIGN 



DESCRIPTION: 



The process of determining the detailed requirements 
for improved information processing and planning of a 
system, plus a set of computer programs capable of 
fulfilling them, is divided into two peurts — System 
Analysis and System Design. The first part, the 
analysis (sometimes called problem formulation), 
consists of investigating the particular information 
processing problem that may be solved by new or 
improved automatic data processing methods; the 
second design consists of attempts to devise a 
satisfactory solution to the data processing require- 
ments involved. In the broadest sense, the problem 
and its solution may involve the design of a far-flung 
network including communications displays, data equip- 
ment for sensors or missiles, computer operating 
procedures, and computer programs. In its narrowest 
sense. Analysis and Design work as part of computer 
programming may only include the design of a change 
to a computer program in an existing system. 



Generally, the mission of the analyzing and synthesizing 
process is to devise the most effective and efficient 
organization of system components including computer 
program functions and elements possible within the 
constraints of available manpower, funds, and time, 
to perform the required information processing 
functions. Ideally, this selection of a solution 
should be made on the basis of cost/benefit compar- 
ison of feasible alternatives, so a prime use of 
the material in this Handbook is to help meinagers 
in making such decisions. 
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TABLE. 



IV 



ACTIVITY DEFINITION (CONT'D.) 



(AFSCM 375 context: Information System Analysis and 
Design begins in the Conceptual Transition Phase, after 
issuance of a Specific Operational Requirement (SOR), 
an Advanced Development Objective (ADO), or an 
Operational Support Requirement (OSR), and ends during 
Phase A, Prepare for Contractor Definition, with the 
issuance of the System Specification. The design 
part of Analysis and Design occurs during Phase B, 
Contractor Definition, The resulting document, a 
firm definition of detailed functions, is equivalent 
to the "Contract End Item Detail Specification 
(Computer Program) — Part I-*') 



TASKS ; 



Analyze System Requirements * Determine the operational 
requirements of the system and evaluate their complete- 
ness, feasibility, and compatibility with other systems 
by studying the original definition of the problem and 
its references and by contacting and coordinating with 
user personnel. 



Analyze the User's Environment * Study the user's 
current environment and operations, to determine hov 
any new system will be employed, where any new operations 
will be located, and what the information processing 
responsibilities of the user are (especially in 
processing inputs from and outputs to other organ- 
izations); and to determine the effectiveness and 
deficiencies of existing data processing operations 
that might be improved by new or changed information 
processing. 

Analyze Computer Program Production Requirements . 
Determine the requirements for computer program pro- 
duction and test, including the adequacy of available 
tools, amount and kind of programming languages, 
operating systems, and other programming support, the 
tools needed to produce any new computer programs, 
existing computer operations, experience with the 
proposed computer, and the projected availability of 
the computer and facility, and the availability of 
back-up equipment. 
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TABLE. 



IV 



ACTIVITY DEFINITION (CONT'D.) 



PRINCIPAL 

INPUTS: 



Analyze Similar and Interfacing Systems « Determine if 
there are systems, subsystems, procedxires, tools, and 
techniques already in production or use or planned, 
that may influence the proposed new system. 

Prepare Design Requirements emd Specifications . Develop 
the specifications for the total information processing 
system including the system configuration that is 
expected to meet requirements. Prepare a functional 
description to serve as a basis for the other develop- 
ment activities and to promote the user's understanding 
of the system. Develop the design requirements for 
the computer program system part of the total information 
processing system. 

Obtain User's Concurrence . By a process of review and 
revision, obtain concxirrence on the system specifications 
ajid the design requirements for the computer program. 

User's requirements including mission statements. 

User's environment including system constraints and 
interfaces, SOPs, and description of existing system. 

Descriptions of the design and operation (including 
experience) for subsystem ajid equipment components. 

Production constraints and environment. 

Orgajiization charts, responsibility charts, ajid job 
descriptions. 

Comparative docxaments (case histories, planning, and 
estimating documents from other similar projects, cost 
studies and reports). 

Description of available tools (to the computer program 
developer) for computer program development such as 
lajiguages, compilers, assemblers, utility systems, 
monitors, test data generation, and recording and 
reduction systems. 

Descriptions of the computers, and other equipment, 
the machine langimge and command structure (repre- 
sentation of instructions and data in the computer). 
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TABLE, 



IV 



ACTIVITY DEFINITION (CONT'D.) 



PRINCIPAL 
OUTPUTS: 



Reports on EAM, computer, and support program usage 
experience and delivery dates. 

Details of computer system to be produced (real time, 
relocatable, task-oriented, cyclic versus stacked job, 
etc. ) . 

Computer operating procedures. 

Requirements analysis — operational and developmental. 

System design specifications including the design 
requirements for the computer programs and the data 
base. 

Plans for system test including the criteria for 
judging performance. 

Replanning inputs — changes to costs and schedules 
obtained from improved knowledge of problem and its 
solution. 

Report on the selection of the system design including 
rationale such as the results of the cost/benefit 
analysis on the alternative ways to accomplish the 
automatic data processing. 
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TARi F V COMPUTER PROGRAMMING COST FACTORS 


FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


♦IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


HANDBOOK 
PAGE 


OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


DESCRIPTIVE 


ANALYTICAL 


Requirements 


+ 




+ 








X - Vagueness of design requirements 
definition 


X^ - Innovation required 


+ 




+ 








X^ - Lack of knowledge of operational 
^ requirements (the problem) 


- 




- 








X. - Number of organizational users 


+ 




+ 








X(- - Number of ADP centers 


+ 




+ 








X_ - Response time requirements 


- 




- 








X^ - Stability of design 


- 




- 








Xq - On-line requirements 


+ 




+ 








X^o - Complexity of system interface 
' with other systems 


+ 




+ 




hk 




Xo^ - Degree of system change 

expected during development 


+ 




+ 








Xo), - Number of functions in the 
^ system 


+ 




+ 








Xqj. - Number of system components 


+ 




+ 








*NOTE: IMPACT IS INDICATED BY: 


+ = VARIES DIRECTLY 

- = VARIES INVERSELY 

NO SIGN = NO PRESUMED DIRECTION 


(FOR DISCUSSION OF CODING, SEE GLOSSARY D) 
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TABLE V : COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 


FACTOR 


*IMPACTON 


REFERENCE 


(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


INDICATED RESOURCE 




OTMCD 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


HANDBOOK 
PAGE 


w 1 n 


iur\ 


DESCRIPTIVE 


ANALYTICAL 


Program Design & Production 


+ 




+ 








X^Q - Number of wordc in data base 


X - Number of classes of items 
in data base 


+ 




+ 




















^20 " ^^^^^^ °^ input message types 


+ 




+ 








X - Number of output message types 


+ 




+ 








Xp2 - Number of input variables 


+ 




+ 






23 


Xp- - Number of output variables 


+ 




+ 








X^K - Number of words in tables and 
constants not in data base 


+ 




+ 




















Xiq - Stringent timing 


+ 




+ 








X. c - Internal docutaentation 


+ 




+ 


W 


57, hh 




X|^ - External documentation 


+ 




+ 








Xt „ 1 ■ "^o^l number of external 
document types written 


+ 




+ 




57, hk 
















X. „ - - Total number of internal 
document types written 


+ 




+ 




57, hk 
















Data Processing Equipment 


+ 




+ 








Xj.- - ADP components developed 
concurrently 














X , - Special display equipment 


+ 




+ 








Personnel 


. 




. 








X^ - Percent senior programmers 


X„Q - Average analyst experience with 
similar systems (see X/-.) 


- 




_ 




















Xg. - Percent programmer design 


- 




. 








participation limit 














Xz-c - Personnel continuity 


- 




- 
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TABLE Y_ 



COMPUTER PROGRAMMING COST FACTORS (CONT'D.; 



FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 



♦IMPACT ON 
INDICATED RESOURCE 



MAN 
MONTHS 



COMP. 
HOURS 



ELAPSED 
TIME 



REFERENCE 



HANDBOOK 
PAGE 



OTHER 



DESCRIPTIVE ANALYTICAL 



Xoy - Percent senior analysts 



^92 



Personnel turnover 



Development Environment 



^67 
^90 



- Lack of management procedures (2 

- Number of agencies concurring 
in design 

- Customer in experience 

- Number of man trips 

- Security classification level 

- Number of sources of system 
information 

- Accessibility of system 
information 

- Number of system components — 
not "off-the-shelf" 

- Quality of resource documents 

- Degree of standardization in 
policy procedures 

- Number of official reviews 
of documents 
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ADDITIONAL COMMENTS ON SELECTED COST FACTORS 



'^h^ 



- Internal Documentation ^ "On the surface, documentation appears to be 
a time-consuming, laborious chore. But when the subject is closely 
examined, it becomes obvious that the bulk of the documentation is 
created as a result of doing a good systems job. This will be true 
unless the documentation requirements are so elaborate that to conform 
with them would be as time-consuming as doing the complete systems 
job." (52) 
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TABLE VI PLANNING FACTORS 


TYPE: VARIOUS SUBJECT: ESTIMATION OF TASKS WITHIN ACTIVITY 


TASK 


ESTIMATION RULE 


Analyze System Requirements 




Analyze User's Environment 




Analyze Computer Program 
Production Requirements 


Up to 9 in£Ln weeks (l8) 


Analyze Similar and 
Interfacing Systems 


Up to 10 man weeks depending on nature of 
project (l8) 


Prepare Design Requirements 
and Specifications 


Estimated at 10-15 percent of total project 
man months, exclusive of maintenance (lo) 


Obtain User's Concurrence 


Three man days per design document per 
agency contacted, plus allowances in elapsed 
time for travel (l8) 
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TABLE. 



VII . 



ACTIVITY DEFINITION 



ACTIVITY: 
DESCRIPTION: 



TASKS: 



COMPUTER PROGRAM DESIGN, CODE, AND TEST 

This activity covers all work necessary to produce the 
computer program end product in accordance with the 
detailed specification of requirements for the computer 
program including design, code, test (debug) and docu- 
mentation work for the entire program as well as 
subprograms (runs, segments, individual programs). 

(AFSCM 375 context: Computer program design occurs 
during the Acquisition Phase, and contributes to the 
"Contract End Item Detail Specification (Computer 
Program) — Part II" and is an input to the Critical 
Design Review (CDR). Computer program coding and 
test occur during the Acquisition Phase and include 
Category I testing and evaluation of the computer 
program by the developer. This step results in a 
complete "Contract End Item Detail Specification 
(Computer Program)— Part II," which is an input to 
the First Article Configuration Inspection (FACi). 
The testing in this activity includes computer program 
functional test that occurs in the Acquisition Phase, 
and is equivalent to Category I Formal Qualification 
testing. For systems developed according to the 
AFSCM 375 series. Category I Formal Qvialification 
testing is usually conducted at the facility 
designated for Category II testing.) 

Develop Program System Test Plans . Develop and 
document program system test requirements, test 
plans, and test designs to provide the specific 
plans and criteria for the computer program system. 
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TABLE. 



VII 



ACTIVITY DEFINITION (CONT'D.) 



Design Programs , Design and docimient the entire 
computer program system and individual programs and 
routines that have been identified. Determine 
input/output message formats. 

Design Individual Program Files . Develop and define 
the form of the data elements to be manipiilated by each 
program, lay out storage allocations, and document 
program data structures. 

Establish Program System Files . Develop and maintain 
a central accounting system for data used by more thaji 
one program in the program system. Document the central 
data file structure and the procediires for maintaining 
it. Periodically issue listings of the central file 
contents. 

Code the Programs . Translate flow diagrams and other 
statements of program designs into coded instructions. 

Desk Check the Programs . Desk check program code by 
looking for ill egad, expressions, erroneous data 
references, program logic errors, program inefficiencies, 
and deviations from program specifications. 

Become Familiar with the Test Environment and Procediires . 
Review the current procediires for using the computer, 
the utility system, and other support systems, using 
the requirements as a guide. 

Compile and Check Program Code . As individual blocks 
or subroutine code are written in either symbolic 
assembly langiiage or procediire-oriented language, 
assemble or compile each block into machine-readable 
(binary) form, check the listings for errors, correct 
the code and recompile. Continue this process until 
a satisfactory compiled program or routine is obtained. 

Test Individual Programs . Within the requirements of 
the plans for program ^sting, run performajice tests of 
the individual programs to isolate and correct errors. 
Rerun tests until all program requirements and design 
specifications are satisfactorily met. 
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TABLE YLL 



ACTIVITY DEFINITION (CONT'D.) 



PRINCIPAL 
INPUTS: 



PRINCIPAL 
OUTPUTS: 



Test Program Subsystems > Within the requirements of 
the plans for program testing, run program subsystem 
tests for physical integration of functionally inter- 
dependent blocks of programs to isolate and correct 
failures to meet program specifications. (Program 
systems consisting of only one individual program 
will not require this task.) 

Functional Test of Complete Program System . Run 
performance and demonstration tests of the complete 
program end product to isolate and correct program 
system malfunctions and demonstrate that the program 
system meets all specifications. (This test is done 
at the developer *s facility, using a simulated total 
information system environment. When user's euid 
developer's facilities are the same (e.g., in-house 
computer progreimmlng on the same computer to be used 
for operations), this task may be omitted.) 

System specifications with computer program require- 
ments, including functional descriptions, system 
configuration, system flow charts, data element 
inputs and outputs, and a description of data base. 

User manuals for computer and associated software 
such as operating systems euid compilers. 

Schedules and budgets. 

Test plans and requirements for computer programs. 

Program system test design. 

Program specifications — both system and individual 
program. 

Broad, detailed flow diagrams. 

Computer input and output formats. 

Coded source program statements (listing). 

Object program in binary, on cards or tape. 
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TABLE . 


WTT : COMPUTER PROGRAAAMING COST FACTORS 








FACTOR 


« 


IMPACT ON 


REFERENCE 




(FOR DEFINITION & CODING, 


INDILaICU KOUUK(-t 




OTHER 




MAN 


COMP. 


ELAPSED 


HANDBOOK 




SEE GLOSSARY A) 










MONTHS 


HOURS 


TIME 


PAGE 


DESCRIPTIVE 


ANALYTICAL 


Requirements 














^1 


- Vagueness of design requirements 


+2B 


+2B 


+2B 


86 






definition 














\ 


- Innovation required 


+2B 


+2B 


+2B 


80, 83, 
86 






^3 


- Lack of knowledge of operational 


+2B 


+2B 


+2B 


77, 85 






requirements 














\ 


- Number of organizational users 


IB 


IB 


+2B 


80, 81 






S 


- Number of ADP centers 


+3 


+2 


+2 


87 






\ 


- Complexity of program interface 


+2B 


+2 


+2B 


80, 82 






with other systems 














h 


- Response time requirements 


+4A 


+4A 


+4A 




13, 21 




h 


- Stability of design 


+4 


+3 


+U 


77-81,83, 
85-87 






^ 


- On-line requirements 


IB 


IB 


+2B 




13, 21, 
25, hk 




hQ 


- Complexity of system interface 


+ 


+ 


+ 
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^3 


- Degree of system change expected 


+ 


+ 


+ 








during operations 














hk 


- Niamber of functions in the 


+ 


+ 


+ 








system 














^5 


- Number of system components 


+ 


+ 


+ 








•NOTE 


: IMPACT IS INDICATED BY: 
CORRELATION SIGN Ol 


rHER CONSIDERATION 


s 






4 - HIGH + - VARIES DIRECTLY A 


-HIGH CORRELATION 


/ 






3 « MODERATE - - VARIES INVERSELY 


BUT ALSO CROSS- 








2 -LOW NO SIGN -NO PRESUMED 


CORRELATION 








1 = INDETERMINATE DIRECTION B 


-STRONG INTUITIVE 


APPEAL 






(FOR DISCUSSION OF CODING, SEE GLOSSARY D) 
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^^^ : COMPUTER PROGRAMMING COST FACTORS (CONT'D,) 


FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


♦IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


HANDBOOK 
PAGE 


OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


DESCRIPTIVE 


ANALYTICAL 


% 


- Nvunber of system components 
not off-the-shelf 


+ 


+ 


+ 








Program Design and Production 














^10 


- Total object instructions 
delivered 


+i^ 


+i^ 


+4 


60 


13 


23 


^11 


- Percent delivered object 
instructions reused 


- 


- 


- 








^12 


- Total nondelivered object 
instructions produced 


+ 


+ 


+ 








h3 


- Total sovirce instructions 
vritten 


+4 


+i^ 


+4 






23 


hk 


- Percen-c source instructions 
vritten in POL 


- 


- 


- 








h5 


- Percent of total object 
instructions discarded 


+ 


+ 


+ 








\6 


- Percent of. total source 
instructions discarded 


+ 


+ 


+ 








\l 


- Number of conditional branches 


+2 


+k 


+2 


85 




23 


\% 


- Number of vords in data base 


IB 


IB 


IB 


80 






h9 


- Nvunber of classes of items 
in data base 


IB 


IB 


IB 


79,81,83, 
86 


13 


23 


^20 


- Number of input message types 


IB 


IB 


IB 






23 


^21 


- Number of output message types 


IB 


IB 


IB 






23 


Xgg 


- Number of input variables 


IB 


IB 


IB 






23 


^23 


- Number of output variables 


IB 


IB 


IB 








^21^ 


- Number of vords in tables and 
constants not in data base 


+ 


+ 


+ 








^25 


- Percent clerical 


-2 


+2 


-2 


82 






^26 


- Percent mathematical 


+2 


-2 


2 


77 






^27 


- Percent l/O 


-2 


-2 


-2 
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TABLE- 


VIII : COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 


FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


*IMPACTON 
INDICATED RESOURCE 


REFERENCE 


HANDBOOK 
PAGE 


OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


DESCRIPTIVE 


ANALYTICAL 


^28 


- Percent logical control 


+2 


+2 


+2 








^29 


- Percent self-checking 


-2 


-2 


-2 








^30 


- Percent infomation storage 
and retrieval 


+2B 


+2 


+2 


77,82,86 






^31 


- Percent data acquisition 
and display 


-2 


+2 


+2 








^32 


- Percent control or regulation 


+2 


+2 


+2 








^33 


- Percent decision making 


+2 


+2 


+2 








^3U 


- Percent transformation 


-2 


-2 


-2 








^35 


- Percent generation 


+2 


+2 


+2 


78,85,87 






^36 


- Average operate time 


IB 


1 


IB 








^37 


- Frequency of operation 

- Insufficient memory 


+4 

+Ua 


+4a 


+4a 


81-83 






X35 


- Insufficient l/O capacity 


+ 


+ 


+ 








""ko 


- Stringent timing requirements 


+ k 


+U 


+ U 


79,81,83, 

8Jt,87 

77,79,80, 

8l,8U,87 

6o,77,8i», 

85 

62 






hi 

\3 


- Number of subprograms 

- Programming language 

- POL expansion ratio 


+ k 
-2 


+4 
-2B 


+4a 

-2 






\k 


- Support program availability 


B 


B 


B 
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^5.1 


- Internal documentation, written 


+4a 


+2 


+3A 




hk, 57 




^U5.2 


- Internal documentation, 
available 


- 


- 












- External documentation 

- Total number of external 
document types, written 


+2B 


+2 


44a 
+2B 


78,82,81tj 

87 

78,79,8it 


hh, 57 




v. 2 


- Total number of internal 
document types, available 


- 


- 


- 
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TABLE jail_ : COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 


FACTOR 

(FOR DEFINITION i CODING, 
SEE GLOSSARY A) 


♦IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


HANDBOOK 
PAGE 


OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


DESCRIPTIVE 


ANALYTICAL 


V.3 


- Total mim'ber of internal 
docvment types, vritten 


+1B 


+18 


+IB 


78,79 






^lv8 


- Type of program 














^h8.2 


- Business 

- Scientific 


-k 

+2B 


-k 

-2B 


-k 

-2B 


77,82,83, 
86,87 






^h8.3 


- Utility 

- Other 


+2B 


+2B 


+2B 


78,79,80, 
83 


10, 50, 

60 




^lv8.5 


- St£ind-alone 


IB 


+2b 


+3b 


77, 80 


13 




^89 


- Availability of special tools 


- 


- 


- 


6k 


58 




X93 


- Output volume 


+ 


+ 


+ 






23 


^Sk 


- Input volume 


+ 


+ 


+ 






23 


Data Processing Equipment 


h 


3 


1 


62 




lU 


^kS 


- Compiler or assembler used 


^50 


- Developmental computer used 


2B 


2B 


2B 








^51 


- First program on computer 

- Average tiirnaround time 


2B 


+4 
2B 


2B 


77,79,81, 

82,83,87 

78,85 






^53 


- ADP coniponents developed 
concurrently 


+4 


+2B 


+4 


62,77,80, 
87 


33 




Xji^ 


- Special display equipment 


+4 


+4 


+2 


80,81,82, 

85,87 
84 
jr7,78,80, 

181,82,85, 
(.87 

78,84,85, 

87 






^55 
^58 


- Core capacity 

- Randan access device used 

- Number of bits in word 

- Memory access time 


+2B 

A. 
-2 


-2 
+4 

+3 
IB 


+2B 
+2B 
■A 
-3A 




23 


X55 


- Machine add time 


-3A 


IB 


-3A 








^60 


- Computer cost 


-k 


-3 


-k 






23 
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TABLE - 


VIII : COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 


FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


* IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


HANDBOOK 
PAGE 


OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


DESCRIPTIVE 


ANALYTICAL 


Personnel 














^61 


- Percent senior programmers 


-UA 


-3A 


-3 


86 






^62 


- Average programmer experience 
with language 


-3 


-IB 


-2B 


63, 85 






^63 


- Average programmer experience 
with application 


-2 


IB 


-2 


81+ 


63 


23 


^61. 


- Percent programmer design 
participation 


IB 


-u 


-3 


78,80,82, 
8I1, 87 






^65 

he 


- Personnel continuity 

- Maximum number of programmers 




+UA 


+UA 


79,82,83, 


38 




hi 


- Percent senior analysts 


- 


- 


- 








X52 


- Personnel turnover 


+ 


+ 


+ 








Development Environment 














hi 


- Lack of management procedures 


-2 


-2 


+2 








^68 


- Niim"ber of agencies concurring 
in design 


+2 


+2 


+2 








h9 


- Customer inexperience 


IB 


IB 


IB 








^0 


- Computer operated by agency 
other than developer 


+2 


+2 


+2 




60 




^1 


- Different site for programming 
and operation 


IB 


IB 


IB 


80 






^2 


- Different computers for 
programming and operation 


+u 


+u 


+u 


63,77,78, 
79,81+ 






^3 


- Closed or open shop 
facility operation 


-3A 


•hk 


-3A 


63 


8, 1+8 




^1. 


- Number of locations for 
program development 


+U 


+3A 


+Ua 


82 






^5 
^6 


- Number of man trips 

- Developed by military 
organization 


IB 






77,79,80, 
83,86,87 
77,87,79, 
80,81,83 
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TABLE 



VIII . 



COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 



FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 



♦IMPACT ON 
INDICATED RESOURCE 



MAN 
MONTHS 



COMP. 
HOURS 



ELAPSED 
TIME 



REFERENCE 



HANDBOOK 
PAGE 



OTHER 



DESCRIPTIVE 



ANALYTICAL 



77 



79 



90 



91 



Developed on time -shared 
computer 

Security classification 

Quality of resource documents 

Degree of standardization 
in policy & procedure 

Number of official revievs 
of documents 



-2B 



-2B 



-2B 



3^ 



52 
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ADDITIONAL COMMENTS ON SELECTED COST FACTORS 
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Total Object Instructions Delivered ^ Obviously, larger programs 
require larger amounts of resources. There is some statistical evidence 
(32) that larger programs also involve a disproportionate increase in 
man months; that is, the man months required per thousand delivered 
instructions increase as the total number of instructions in the program 
increases. That a program with many instructions costs more per 
instruction than one with fewer instructions is a commonly held belief 
(13, 36). One reason that this may be so is that leu'ger programs must 
usually be broken down into smaller pieces for individuals to work in; 
this complicates both the program design problem and the subsequent 
assembly and checkout of the various subprograms (27). The evidence 
on this point is, however, not conclusive. 



^U2 



Programming Language . One authority states the following on the general 
question of procedure -oriented language versus assembly language 
programming (55)- 

"a) In general, there is no appreciable difference in the amount of 
training needed to acquire professional competence in the use of 
either a procedure langtxage or an assembly language. 

"b) The use of an appropriate procedure language, by reducing the 
number of steps in the source program and by easing the job of 
program modification, can significantly reduce the amount of 
effort needed for program production and maintenance. 

"c) The use of a procedure language instead of an assembly language 
improves the communication of algorithms between programmers- - 
primarily by reducing the number of steps needed for their 
expression. This improvement resiilts in a reduced need for 
detailed, step-by-step program documentation. 
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"d) Procedure languages, beca\ise they are largely machine independent 
and because they make programs easier for programmers to read and 
to modify, can greatly facilitate the transfer of programs between 
different computer types. Of course, the transfer of programs 
that are system-dependent is not always practical, however machine- 
dependent, readable, and changeable they are. 

"e) Object-code efficiency is often not an important factor; neverthe- 
less, with a good compiler, an average programmer will usually turn 
out code that is Just about as efficient as the code he would 
produce using an assembler. 

"f) The use of a procedure language instead of an assembly language 
does not increase the difficulty of obtaining program timing 
estimates, since these estimates seldom involve the execution 
times of individual program steps." 

A definite trend from machine-oriented languages to problem-oriented 
languages is frequently cited in the literature (7). 

The resource cost rates cited in this Handbook provide, for the specific 
sample of programs studied, some evidence of the relative overall expen- 
diture of resources for the total program design, code, and test effort 
for several procedure-oriented langioages. For source-program compilation 
time only, a test was made (IBM 709O with IBSYS) with four programs 
written in both FORTRAN and COBOL (?)• The following results of this 
test indicate longer compile times for COBOL: 



Number of Cards 
in Source 
Type of Problem Program 


Compilation 

Times 
(in minutes) 


^ Increase in 
Compilation 
of COBOL 
over 


FORTRAN 


COBOL 


FORTRAN 


COBOL 


FORTRAN 


Job Order Cost l8 
Calculation 


72 


0.5007 


0.7989 


59.6 


Sort Routine 25 


56 


0.4890 


0.8108 


65.8 


Hourly Payroll 35 
Computation 


115 


0.5099 


0.8390 


64.5 


Order Processing 22 
Cycle 


288* 
report 


o.5o4o 

on rejected 


1.1111* 
orders . 


120.46* 


♦Includes additional output 
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POL Expansion Ratio * Working with the better JOVIAL compilers, the 
size of the object program generated from JOVIAL source statements 
may be 10-15 percent larger than if the source program had been coded 
in machine languages (62). Likewise, in terms of size of the object 
program, COBOL programs average 10-15 percent larger than their 
machine langixage counterparts (37) • This "excess generation" would 
be a part of the POL expansion ratio, since this ratio is computed by 
dividing the total number of object instructions by the total number 
of source language statements. 



X. Q - Compiler or Assembler Used * The impact of the compiler-hardware 

combination on compilation cost is illustrated by the following (lU); 



Computer 


Typical 

Compilation 

Speed, COBOL 

Statements 

per minute 


Prime Shift 

Cost per 
minute ($) 


Cost per 

100 COBOL 

Statements ($) 


IBM- 7010 


550 


1.88 


.■^ 


H-1800 III 


1000 


2.31 


.23 


B-5000 


U50 


2.15 


.U8 


U-1107 


700 


3.50 


.50 


IBM-7OTU 


30 


2.66 


8.87 


IBM 7080 


29 


5.02 


17.31 


H-UOO 


22 


.78 


3.55 


RCA- 301 


12 


.5h 


U.50 


IBM-IUOI 


10 


.52 


5.20 
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- ADP Components Developed Concurrently . Experience by the USAF in the 
development and programming of special-purpose computers such as 
AN/fSQ-T, AN/fSQ-8, AN/fSQ-31, AN/fSQ-11, has revealed that the 
checkout of new unproven hardware from one company and the unproven 
software (see Information System Integration Test Activity, Section VI) 
to make it perform from another can add significantly to the expense 
of the project. A major problem is the difficulty in establishing 
responsibility when either hardware or software fails to perform to 
specifications {2h)» 
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X/'p - Average Programmer Experience vlth Langueige , "A knowledge of the 
'compiler's ways' is invaluable to the user and can account for 
differences of as much as 40^ in running time, and even more in 
object program size." (37) 

yirr - Maximum Number of Programmers . Absolute values of the resource units, 
e.g., number of computer hours, correlate directly with the maximum 
nimiber of programmers assigned (Xgg), primarily because more programmers 
must be assigned to the larger development efforts in order to attempt 
to meet the imposed schedules. It should not be inferred from this, 
however, that larger programming staffs are inherently less efficient, 
e.g., have lower production rates, than smaller groups. If the staff 
is properly managed, there are no a priori reasons why a large staff 
shoxild not operate as efficiently as its smaller counterpart (38) . For 
a contradicting view on this point, see (58). 

X,p - Different Computers for Programming and Operation . An advantage claimed 
for problem-oriented languages is that a program written for one computer 
can be readily recompiled to run on another. But there are still costs 
of making this conversion. "Tests show that converting a COBOL source 
program from one computer to another involves only 10 to h<y^ of the time 
required to write the original program in COBOL. It should be stressed 
that this time is a percentage of actual 'coding' time, and not the 
total time required to analyze, design and code. If the two machines 
are very similar in design and have the same collating sequence, the 
original coding time might be reduced by 95^* " (37) 

X^^ - Closed or Open Shop Facility Operation . Arguments can be made for either 
type of facility operation (8). The following advantages are claimed 
for the "open shop": 

1. A programmer can discover more errors per test shot. 

2. By observing the program run, the programmer may be able to detect 
operational weaknesses in the program. 

3« Programmers may be better machine operators, and be more careful 
with their own programs, resulting in fewer operator errors during 
testing. 

U. Testing time is reduced because of quick turnaround. 

On the other hand, the following arguments are advanced in favor of the 
"closed shop" : 

1. The programmer may foul up his program by experimenting with the 
console after a hang- up. 
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2. Programmers waste time waiting for machine availability, and may be 
less efficient than a regular operator getting on and off the machine. 

3. Programmers tend to waste machine time. 

k. If test instructions are prepared for operators, the programmer tends 
to organize test shots better. 

The research of this project is not conclusive, but computer hours per 
1000 object instructions tend to be larger in open-shop operations; 
correlation coefficient of computer hours per 1000 object instructions 
with variable X'jo (coded: open shop = 0, closed shop = l) is -.21 for 
the total sample of 169 data points. From this statistic, one infers 
that computer usage rates are lower in closed shops. 

In a System Development Corporation survey of 30 organizations (3^); "the 
following describes the tendency to favor the closed-shop system: 





Scientific 


Business 






Programming (^) 


Programming (^) 


Both (?^) 


Open Shop 


39 


12 


27 


Closed Shop 


61 


88 


73 
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- Time-Sharing . In the first known study of the relative performance of 
prograanmers working under conditions of on-line (time-sharing) and off- 
line access to computers, the following conclusions were suggested (3^)t 

1. On-line operations are superior to off-line in terms of man hours 
spent debugging. 

2. On-line operations tend to use more computer time than off-line 

programming. 



3. There are striking individual differences in programmer performance. 

Xqq - The Availability of Special Tools . It has been claimed that experience 

at General Electric shows that the actual time spent in problem definition 
and coding can be reduced with the use of decision tables. "Users report 
up to 90^ lower programming costs because detailed flow charting and 
coding are virtually eliminated. Other users report 50^ lower applica- 
tion costs because of this, plus improved systems design and check-out 
efficiency/' (52) 



6k 




TABLE i2L 



PLANNING FACTORS 



TYPE: 



VARIOUS 



SUBJECT: 



QUOTES WITHOUT COMMENT 



1. "A programmer can prepare, check out, debug, and document about 1 
Instruction per hour." (36) 

2. "In the past, preparation costs of computer programs averaged $8 
per instruction. . .nev techniques have cut this cost to about $.80 
per instruction. " (^3) 

3. "For large scale programs, approximately 50 instructions can be 
developed per hour of computer time.'* (36) 

k. "The (corripilers) in existence today have taken on the order of 

15 to 25 man years of experienced programming talent, spread 
over about three calendar years." (59) 

5. "The charges for direct salaries and supplies approximately equal 

to first shift rental of the computer in an average installation." (45) 

6. "Less than 5^ of the total cost of a computer center is in coding." (12) 

7. "For real-time systems, of the time betveen the appearance of the 
first vinit function-statements and the beginning of program 
acceptance tests, at least 2/3 should be allotted to build-up 

and checkout of successive 'packages* culminating in the completed 
program; and an 'executive* routine, including control of test inputs 
and recording, should be ready, together with the first nucleus of 
program units, no later than the end of the first third of this 
interval." (25) 
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TARIF HII 



PLANNING FACTORS 



TYPE: 



SUBJECT: 



Unit Cost 



Man Months Expendltxire Hates 
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Percent of Totca Saorple, N«l69 



Note : ThlB curve Is a plot of the man month expenditure rate per ICXX) object instruc- 
tions (ordinate) against the percent of the total sample of ccmpleted cauputer programs 
(absissa) that experienced this rate or less. The typical range is defined to exclude 
the upper and lover 20 percentiles. Example: if the estimator believed his program 
is more "difficult" than the median (50^ on the ahsissa) hut not atypical; i.e., as 
"difficult" as the extreme values, he might choose to use rates for the 6O-8O percent 
range; then his expected expenditure rate taken from the ordinate vould he 3*9-6.3 
man months per 1000 object instructions. 



69 




TAttI P XIV . PI AMMIMn FArTnR<; 


TYPE: SUBJECT: 

UNIT COST ESTIMATION OF TASKS WITHIN ACTIVriT 


TASK 


RULE FOR ESTIMATING MAN MONTHS 


Develop Program System Test Plans 


1 man month per 10,000 estimated 
machine instructions (l8). 


Design Programs 


1/2 to 1 man month per 1000 machine 
language instructions. 

1 man month per 1000 machine language 
instructions for programs > 30,000 
instructions (I8). 


Design Program Files 


1 man month per 10,000 items (I8) 


Establish System Files 


1 man month per 10,000 machine language 
instructions. 

2 man months per 10,000 machine language 
instructions for programs > 30,000 
instructions (18). 


Code Programs 




Desk Check Programs 




Learn Test Environment and 
Procedures 


1 man veek per programmer 


Compile and Check Program Code 




Test Individual Programs 


See page 71* 


Test Program Subsystems 


See page 71* 


Functional Test of Complete 
Program System 


See page 71. 
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TAfti p XV . PI AMMiKir; PArrnp*; 


^^^' PERCENT OF OTHER SUBJECT: ESTIMATION OF TASKS WITHIN ACTIVITY 
ITEM 


TASK 


RULE FOR ESTIMATING MAN MONTHS 


Develop Program System Test Plans 


See x>age 70. 


Design Programs 


See page 70. 


Design Program Files 


See page 70' 


Establish System Files 


See page 70. 


Code Programs 




Desk Check Programs 




Learn Test Environment and Procedures 


See page 70. 


Compile and Check Program Code 




Test Individual Programs 


All testing requires iiO-50 percent of total 
development effort (I8). 

Program test reqixLres about 20 percent of 
testing effort. 

Expect one error per 30 instructions (I8). 


Test Program Subsystems 


All testing requires iiO-50 percent of total 
development effort (I8). 

Subsystem testing requires O-3O percent of 
testing effort, depending on niimber of 
subsystems (I8). 


Functional Test of Complete 
Program System 


About 50 percent of total testing effort, 
hence, about 20-25 percent of total development 
effort (18). ' 
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TAftI F XVI . PI AMhsllKin FArTOR<; 


^^^' PERCENT OF OTHER ITEM ^^^J^^^' PROGRAM DEVELOPMENT COMPUTER TIME 








Percentage of Work 

Processed by Computer in 

27 Firms in Southern California* 


Scientific 


Business 


Both 


Computer time spent in 
developing or modifying 
programs used for a 
specific purpose in 
future computer 
applications 


46 


30 


to 


Production operation of 
checked-out programs for 
specific tasks, e.g., 
payroll, data analysis 


47 


66 


55 


Computer time spent in 
developing programs for 
general applications or 
system control 


7 


k 


5 


TOTAL 


100^ 


locyf, 


100^ 


♦Source: (3^) 
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TABIF XVII ■ 



PLANNING FACTORS 



TYPF> *ilJB JECT* 

PERCENT OF OTHER ITBM '^^"-'^^'* ESTIMATING CCMPUTER HCXJRS FROM MAN MONTHS 



2000 




NOTES: 



1) ag is measured parallel 
to the ordinate. 

2) Computer size Is as 
defined by computer 
cost, X^. 

3) Dashed lines represent 
extrapolation beyond 
range of existing data 
base. 



20 40 60 ttO 100 120 llO 160 180 200 220 21*0 260 280 300 

Man Months 
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COMPUTER PROGRAMMING COST-ESTIMATING EQUATIONS 



The following pages contain several cost -estimating equations for computer 
program design, code, and test. These equations were derived by statistical 
analysis of the total sample of I69 data points, as well as of selected sub- 
samples from the total sample; they represent the best predictors of the cost 
of the programs in this data base that the Programming Management Project was 
able to produce with the available resources. 

In all, three subsamples were investigated: programming language ('Xl^2)f "type 
of application (X1^.8)^ and computer size based on cost (X6o)» These subsamples 
were further broken down into: MOL and POL; business programs (predominantly 
file- oriented, Xl4.8,i); scientific programs (predominantly process-oriented, 
^if8.2)^ software, i.e., computer utility programs (Xl^.8,3); *^^®^ programs that 
could not be classified as business, scientific, nor utility, such as command 
and control (Xi^8,U)> independent, or stand-alone programs (Xl^.8,5), ^^^ programs 
prepared using a large-, medium-, and small-scale computer, as defined by an 
arbitrary cost range (X6o)» For each of these ten subsamples, attempts were 
made to develop prediction equations for each of the three basic resources: 
man months, computer hours, and months elapsed. Equations for the total sample 
(N = 169) were developed first; then attempts were made to produce equations 
for subsamples that represented an improvement over the total sample. The 
criteria for improvement included increased (over earlier equations in refer- 
ences (19^ 31^ cind 62)) intuitive appeal as well as increased statistical 
precision over the equations in the total sample. The measures of statistical 
precision include low standard error of estimate (^), especially when related 
to the mean, and the standard deviation (o) of the resource (i.e., man months); 
high correlation coefficient (r) and coefficient of determination (r^). For 
those subsample equations that showed some improvement and were included in 
this Handbook, Table XVIII simmarizes their statistical characteristics, e.g., 
sample size, mean, and standard error of estimate. (These statistics are 
defined in Glossary B. ) 

The development and improvement of cost estimating relationships is an iterative 
procedure (20, 62). A large number of combinations of subsamples and variables 
are possible, and many analytical trials may be necessary before the more useful 
and meaningful results emerge. Additional experimentation with the existing 
169-point data base could include various other attempts at subsample definition, 
such as by size of program (3l)^ by make of computer hardware, or, for that 
matter, by subsamples based on most of the variables in Glossary A; another 



7h 



prcmlBlng project would be an attempt to produce predictOTB of computer pro- 
gramming expenditure rates such as man months per 1000 source statements as 
characterized In the Planning Factors for this section. 

Before using the following equations, the reader's attention is called to the 
coding of all variables in Glossary A, In addition, to help the reader identify 
the numbered variables in the equations quickly, the names of these variables 
have been listed with their numbers in a foldout lirauedlately following the last 
equation in this Section. The selection by the Handbook user of a particular 
equation will depend on the appropriateness of the equation to his problem, 
and his knowledge of the values of the Independent variables. 
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TABLE 



XXX . 



ACTIVITY DEFINITION 



ACTIVITY: 
DESCRIPTION: 



TASKS: 



INFORMATION PROCESSING SYSTEM INTEGRATION TEST 

This activity covers all work necessary to test the 
performance of the computer program within the total 
system at the operational facility iznder realistic 
("live") operating conditions. 

(AFSCM 375 context: This activity occurs in the 
Acquisition Phase and is equivalent to Category II 
testing. ) 

Conduct Test . Within the requirements of the plans for 
program testing, conduct a sequence of tests of the 
total operational system, receiving actual data from 
and transmitting actual data to other subsystems and 
components that constitute the system. 

Analysis of Test Results . Determine if the system 
meets specifications, and study operations for any 
evidence of potential difficulties. Coordinate with 
other subsystem and component developers to isolate 
sources of poor system performance. 

Initiate Modifications to Computer Programs . Correct 
errors in programs and associated documentation. Design 
and implement feasible changes to meet specified 
performance requirements. This involves work in the 
earlier activities. 

Documentation of Test Results . Prepare appropriate 
compliance documentation to certify successful tests. 
If problem areas arise, document the evidence for use 
in the modification process. Identify potential 
improvements that could be made to the total system. 
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TABLE 



xzz 



ACTIVITY DEFINITION (CONT'D.) 



PRINCIPAL 
INPUTS: 



PRINCIPAL 
OUTPUTS: 



Information system test plans. 

Information system specifications. 

Program system design specifications. 

Program system symbolic deck, bineury deck, listing. 

Operating system data. 

Descriptions of the design and operation of other 
subsystems and components. 

Program system test compliance documentation. 

Modifications including error corrections and changes 
for the computer programs and their documentation. 

Recommendations for future modifications. 
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TABLE JJi^LL COMPUTER PROGRAMMING COST FACTORS 


FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


♦IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


HANDBOOK 
PAGE 


OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


DESCRIPTIVE 


ANALYTICAL 


Re<^uirements 


+ 


+ 


+ 








Xi^ - Number of orfranizational users 


Xj. - Number of ADP eenters 


+ 


+ 


+ 








Xg - Stability of design 


+ 


+ 


+ 








Xg»^ - Number of funetions in the 
system 


+ 




+ 








Xqc - Number of system eomponents 


+ 




+ 








Xo/- - Number of system eomponents 
not off-the-shelf 


+ 




+ 








Program Design & Produetion 




+ 








23 


X^Q - Total objeet instruetions 
delivered 


X,_ - Number .of eonditional branehes 




+ 










X, Q - Number of words in the data 
base 




+ 








23 


X-g - Number of elasses of items in 
the data base 




+ 








23 


XoQ - Number of input message types 




+ 








23 


Xp, - Number of output message types 




+ 








23 


X^^ - Number of input variables 




+ 








23 


*NOTE: IMPACT IS INDICATED BY: 


+ = VARIES DIRECTLY 

- = VARIES INVERSELY 

NO SIGN = NO PRESUMED DIRECTION 


(FOR DISCUSSION OF CODING, SEE GLOSSARY D) 
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TARIF XXXI : COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 


FACTOR 


* IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 




OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 

TIME 


HANDBOOK 
PAGE 


DESCRIPTIVE 


ANALYTICAL 


Xp - Number of output variables 




+ 










X^ - Average operate time 




+ 










X|^2 •* Programming language 














X, - Internal documentation vritten 


+ 


+ 


-»- 








^Us 2 ' ^^"^^^^^1 documentation 
available 














- 


- 


- 








Xy^ - External documentation 


+ 


+ 


-»- 








X. - Total number of external 
document types written 


+ 




+ 




















X. „ 2 ' "^^"^-^ number of internal 
document types available 


- 


- 


_ 




















X, ^ - Total number of internal 
document types vritten 


+ 




+ 




















X|^ - Type of program 














Data Processing Equipment 














X(.^ - First program on computer 


-»- 


+ 


-»- 








X 2 " Average turnaround time 






+ 








X ^ - ADP components developed 
-^^ concurrently 


+ 


-»- 


-»- 




















X , - Special display equipment 


+ 


+ 


+ 








X(.(. - Core capacity 




- 








23 


X /- - Random access device used 




+ 










X(.„ - Number of bits per vord 




- 










X(.Q - Memory access time 




+ 










XcQ - Machine add time 




+ 










X/-^ - Computer cost 












23 



9^ 



TABLE XXXI : COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 


FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


NMPACTON 
INDICATED RESOURCE 


REFERENCE 


HANDBOOK 
PAGE 


OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


DESCRIPTIVE 


ANALYTICAL 


Personnel 














Xg^ - Percent senior programmers 


- 


- 


- 








X/'p - Average programmer experience 
vith language 


- 


- 


- 








Xg- - Average programmer experience 
^ vith application 


- 


- 


- 






23 


X^r- - Personnel continuity 
55 


- 


- 


- 








Xo„ - Percent senior analysts 


- 


- 


- 








Xqp - Personnel turnover 


+ 


+ 


+ 








DeveloTxnent Environment 














Xg„ - Lack of management procedures 


+ 


+ 


+ 








Xx-o - Number of agencies concurring 
^^ in design 


+ 


+ 


+ 








Xj^ - Conputer operated by agency 
' other than program developer 


+ 


+ 


+ 








X,^- - Program developed at site 
' Cfther than the operational 
installation 


+ 


+ 


+ 








X,p - Different computers for 
' programming and operation 


+ 


+ 


+ 








X^Q - Security classification level 


+ 




+ 








Xoo - Quality of resource documents 


- 


- 


- 








Xoq - Availability of special tools 


- 


- 










Xq-j - Degree of standardization in 
'^ policy and procedures 


- 


- 










Xq^ - Number of official reviews 
of documents 


+ 




+ 
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TABLE^QLLL 



PLANNING FACTORS 



TYPE: 



PERCENT OF 
OTHER ITEM 



SUBJECT: COSTING SYSTEM INTEGRATION TEST 



Page 57 of Reference (l8) shows the following average percentages 
for the relative division of costs: 



Activity 
Analysis and Design 
Program Coding and Checking 
Checkout and Test 



jo of Total Project Costs 

3^.5 
18 

^7.5 



The activity labeled Checkout and Test would include the Infor- 
mation System Integration Test activity defined in this Handbook. 
It is estimated that integration test would range from 0^ to 30^ 
of the total cost for computer programming depending upon (l) the 
number of components and subsystems in the total information system 
(2) how new these are and (3) how thoroughly any new component or 
subsystem had been tested prior to the integration test. 
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TABLEUdl: 



ACTIVITY DEFINITION 



ACTIVITy: 
EESCRIPTION: 



TASKS; 



INFORMATION PROCESSING SYSTEM INSTALLATION AND TURNOVER 

The purpose of the turnover step is to help the user 
demonstrate, at his own operational site, that the 
computer program system will operate as specified, and 
to support the user with documentation, advice, and 
guidance, and trouble shooting during the initial period 
of system operation. 

(AFSCM 375 context: Information System Installation 
and Turnover occurs in the Acquis it ion- Operational 
Overlap Phase, beginning with the installation of the 
information processing contract end- item at an 
operational site other than the Category II site, and 
ending vith turnover of the Information system to the 
user at the last operational site.) 

Verify Program System Specifications . Verify the accuracy 
and completeness of program system specifications and 
other documents. Produce any modifications needed. 

Prepare User Documentation . Determine user documen- 
tation requirements and, if necessary, issue 
specifications for this documentation. Prepare 
documentation production plan. Perform the technical 
writing and editing necessary to produce user 
documentation, coordinate the production of material 
by all contributors, and verify the adequacy and 
accuracy of the results. Obtain user approval and 
concurrence on documentation. 

Advise User on Data Conversion . Assist user in 
preparing data collection and conversion plans; advise 
on the technical aspects of the task and the adequacy 
of results. 
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TABLE 



XXXIII 



ACTIVITY DEFINITION (CONT'D.) 



Develop User Training Plan . Determine training 
requirements including: characteristics of those to 
be trained; present and future duties; extent and 
content of training required. Assist user in 
developing a curriculum and schedule. Prepare course 
materials and visual aids required. 

Conduct Training Program . Conduct classes, briefings, 
or demonstrations to train user personnel to interpret 
and prepare inputs and outputs, and to control and 
maintain the computer and the program system. 

Conduct Demonstration Test . Produce or assist in the 
production of system test material required for the 
demonstration test. Brief participating personnel 
on the objectives and procedtares of the demonstration. 
Dry- run the test. Conduct the test jointly with user 
personnel, and if necessary document the results. 
Conduct debriefing sessions after demonstration tests. 

Assist in Operational Shakedovm . Monitor initial 
period of operation of the system to detect and 
correct malfunctions and inefficiencies, and to 
evaluate performance. Establish procedures for the 
user to report suspected program malfunctions. 
Identify reported malfunctions as either: 

(1) error in programming (requirements understood 
but incorrectly implemented) 

(2) error in logic (requirements not understood) 

(3) operator error 
(h) equipment failure 

and take corrective action. 



PRINCIPAL 
INPUTS: 



System functional description and specifications 
Program system design documentation 
Design change documents 
Program flow charts 
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TAni F XXXIII ■ 



ACTIVITY DEFINITION (CONT'D.) 



PRINCIPAL 
OUTPOTS: 



Index to and descriptions of cooiputer program 
documents 

Source program listings 

Object programs 

Prior system documentation 

Data base design and data definition documents 

User documentation 

User training plan and material 

System test material 
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TARi F XXXrV COMPUTER PROGRAMMING COST FACTORS 


FACTOR 


*IMPACTON 
INDICATED RESOURCE 


REFERENCE 




OTHER 


(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


HANDBOOK 
PAGE 


DESCRIPTIVE 


ANALYTICAL 


Requirements 














X - Vagueness of design 

requirements definition 


+ 


+ 


+ 








Xp - Innovation required 


+ 


+ 


+ 








X^ - Lack of knowledge of 

operational requirements 


+ 


+ 


+ 








X, - Number of organizational 


+ 


+ 


+ 








users 














Xj- - Number of ADP centers 


+ 


+ 


+ 








X^ - Conrplexity of program system 
interface 


+ 


+ 


+ 








Xq - On-line requirements 














X„Q - Complexity of system 

interface vith other systems 


+ 


+ 


+ 








Xq^ - Degree of system change 

expected during operations 


+ 


+ 


+ 








Xn. - Number of functions in the 
system 


+ 


+ 


+ 




















Xq - Number of system components 


+ 


+ 


+ 








Xq^ - Number of system components 
not off-the-shelf 


+ 


+ 


+ 








*NOTE: IMPACT IS INDICATED BY: 




^ = VARIES DIRECTLY 

- = VARIES INVERSELY 

NO SIGN = NO PRESUMED DIRECTION 




(FOR DISCUSSION OF CODING, SEE GLOSSARY D) 
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TABLE XXXIV : COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 


FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


♦IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


HANDBOOK 
PAGE 


OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


DESCRIPTIVE 


ANALYTICAL 


Program Design & Production 














X - Total object instructions 
delivered 


+ 


+ 


+ 






23 














X - - Total source instructions 
■^^ vritten 


+ 


+ 


+ 






23 














X, 1^ - Percent source instructions 
vritten in POL 


- 


" 


_ 




















X „ - Nxuiiber of conditional branches 


+ 


+ 


+ 








X^ o - Number of words in the data 
^^ base 


+ 


+ 


+ 






23 


X Q - Number of classes of items 
^ in data base 


+ 


+ 


+ 






23 














XpQ - Number of input message types 


+ 


+ 


+ 






23 


Xp - Number of output message types 


+ 


+ 


+ 






23 


X^^ - Niimber of input variables 


+ 


+ 


+ 






23 


Xp- - Niimber of output variables 


+ 


+ 


+ 








Xp, - Number of words in tables, 

and constants not in data base 


+ 


+ 


+ 




















X^ - Average operate time 




+ 










Xr, - Number of Subprograms 


+ 


+ 


+ 








X, p - Programming language 














X|^j. - - Internal dociimentation written 


+ 


+ 


+ 








X|^j. P - Internal documentation 
'^' available 


- 


- 


- 




















X|^ - External documentation 


+ 


+ 


+ 








X|^„ , - Total number of external 
dociiment types written 


+ 


+ 


+ 




















Xi „ 2 " Total niimber of internal 
document types available 


. 


. 


. 
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TABLE ^™^^ : COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 


FACTOR 


♦IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 




OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


HANDBOOK 
PAGE 


DESCRIPTIVE 


ANALYTICAL 


X, - - Total number of Internal 
document types written 


+ 


+ 


+ 




















X^n - Type of program 














Xq^ - Output volume 


+ 


+ 


+ 






23 


X , - Input volume 


+ 


+ 


+ 






23 


Data Processing Equipment 














X - Developmental corrrputer used 














X^ - ADP corrrponents developed 
concurrently 


+ 


+ 


+ 




















X 1^ - Special display equipment 


+ 


+ 


+ 








Xcg - Random access device used 




+ 










X - Number of bits per word 




- 










Xj-g - Memory access time 




+ 










X - Machine add time 




+ 










XgQ - Coniputer cost 


+ 


+ 


+ 






23 


Personnel 














Xg^ - Percent senior programmers 


- 


- 


- 








Xg2 " Average programmer experience 


- 


- 


- 








vlth language 














X^-3 - Average programmer experience 
^ vlth application 


- 


- 


- 






23 














X^ - Personnel continuity 


- 


- 


- 








Xg - Percent senior analysts 


- 


- 


- 








X p - Personnel turnover 


+ 


+ 


+ 
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TARIP XXXIV . 



COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 



FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 



♦IMPACT ON 
INDICATED RESOURCE 



MAN 
MONTHS 



COMP. 
HOURS 



ELAPSED 
TIME 



REFERENCE 



HANDBOOK 
PAGE 



OTHER 



DESCRIPTIVE ANALYTICAL 



Development Environment 



ho 



71 

^90 
V 



- Lack of management procedures 

- Number of agencies concurring 
in design 

- Customer inexperience 

- Computer operated by agency 
other than program developer 

- Program developed at site 
other than the operational 
installation 

- Different computers for 
programming and operation 

- Security classification level 

- Quality of resource documents 

- Availability of special tools 

- Degree of standardization in 
policy and procedures 

- Number of official reviews 
of documents 



10^^ 
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ADDITIONAL CCMIENTS ON SELECTED COST FACTORS 



^9 



The Availability of Special Tools . In the information system turnover 
activity, particiolarly in the data conversion task, the availability of 
special tools coiold be considered to include special input devices such 
as magnetic ink character recognition, or optical character recognition 
(51). The economics of such devices are obviously a function of the 
volume of data to be processed (note that for this activity the Job is 
to build the working data base, not process incoming data on ein ongoing 
basis). For optical character recognition, one study estimated the 
breakeven point for the Recognition Equipment, Inc., Electronic Retina 
Computing Reader to be a monthly data volume of about a half million 
card equivalents, where the alternative is keypunching with 100 percent 
verification (U7). 



IQlt 



I ' 1 




TAftiF XXXV . PiAMWiKir, PArrnp^ 


"^^^* VARIOUS SUBJECT. COSTING DEMON S^i'RATION TESTS 


TASK 


COSTING FORMULA 


Conduct Demonstration Test 


Final preparation and actual conduct of the 
test for a program system of moderate size 
should take an elapsed time of about one 
veek. One or more dry runs, especially if 
associated vlth operator training, may add 
one or more veeks to the schedule. 

Estimate about two man weeks for a program 
system of about 10,000 to 20,000 
Instructions. 


Analysis of Test Results 


Allow about one man week for system of 
about 10,000 to 20,000 instructions. 


Documentation of Test 
Results 


Allow two man weeks for drafting test 

report . 


NOTE: When a formal program system demon- 
stration test is not required, as is some- 
times the case vhen programs are operated 
and developed by the same organization, 
this activity is covered by the "Functional 
Test of Complete Program System" task in the 
previous activity, computer program design, 
code, and test. 





105 




TABLE XXXVI , 


PLANNING FACTORS 


TYPE: uijjT COST SUBJECT: ESTIMATION OF TASKS WITHIN ACTIVITY 


TASK 


COSTING FORMULA 


Verify Program System 
Specifications 


Technical review: 20 pages per man day 
Revise: 10 pages per man day 




Collect information: 2 days per document 
plus 2 hoiirs per interview 




Type: 20 pages per man day 




Expect at least two drafts of revised 
specifications (l8) 


Prepare User Documentation 


Outline: 2 man weeks per user's docvmient 




Drafting rate: 3-5 pages per man day 




Technical review: 20 pages per man day 




Edit: 50 pages per man day 




Revise: 10 pages per man day 




Type: 20 pages per man day 




Illustrate: 2 illustrated pages per man day (i8) 


Data Conversion 


Keypunching: Key Strokes Per Ho\ir 

Including Set-Up (8) 




All alphabetic 5,000 
punching 




Mixed alphabetic- 4,000 
nvuneric 




Mostly numeric, little 8,000 
alphabetic 




All numeric punching 10,000 




Keypunching or key verification generally 
runs $1.00 per 1000 card per card colvimn (56). 
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TA..PX20CVII. 



ACTIVITY DEFINITION 



ACTIVITY: 
DESCRIPTION: 



COMPUTER PROGRAM MAINTENANCE 

Computer progreun maintenance is the process of improv- 
ing, changing, and correcting computer progreuns in an 
information system that is cixrrently operational. 

Program maintenance, including both revisions and 
error corrections, is needed throughout the life of 
the information system. Revisions are needed because 
operational requirements are continuaJJLy changing 
diiring both the development and operation of the 
system. Although operational needs are projected 
diiring requirements analysis, in most cases they can 
be neither totally defined nor totally implemented 
in the imposed time schedules. Also, corrections 
must usually be made to the computer programs because 
errors and operational deficiencies not detected in 
the routine testing of the programs are usually 
discovered when the system becomes operational. 

Since the need for improvement and support activities 
for the information system tends to be amorphous, 
system maintenance is often fionded at a level the 
user can afford or is willing to spend rather than 
the level precisely required. Much of the work of 
program maintenance personnel must be devoted to 
the resolution of emergencies; a good share of the 
remainder, to modifications required by hard-to- 
predict environmental changes. 



107 



TABLE r^y^^^ 



ACTIVITY DEFINITION (CONT'D.) 



TASKS: 



Develop a Maintenance Plan and Organization * Estimate 
level of maintenance activity required for the compu- 
ter program system. Establish the amount of surplus 
resoiirces desired to Insure the proper handling of 
emergency situations. Determine responsibility and 
authority for the maintenance function. Procizre and 
train necessary personnel. 



FRU^CIPAL 
INPUTS: 



Provide Communications Betveen User and Computer 
Program Developer . Establish and maintain formal 
communications between the information system user, 
the computer program developer, and other interacting 
agencies. 

Establish Internal Communication Channels . Provide 
internal communication channels for processing informa- 
tion flow between various operational sites. Provide 
for periodic progress and activity reports. 

Establish Change Procedizres . Establish procedixres for 
installing error corrections and system retrofits into 
the operating computer program system. Provide proce- 
diires for updating computer program and information 
system documentation. 

Process System Design Changes . Whether design changes 
emanate from reqiiirements changes, or the detection of 
program error, generally this task involves all of the 
preceding steps in the programming process. 

Provide User Support . Provide consulting support on 
management problems, information system hardware, data 
collection and conversion, and system operation, as 
required by the user. 

Computer program documentation, including design 
specifications and descriptions, source program 
listings, and user manuals. 

Design change requirements. 

Information system specifications. 

System malfunction reports. 
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TABLE XXXVII. 



ACTIVITY DEFINITION (CONT'D.) 



PRINCIPAL 
OUTPUTS: 



Revised program code. 

Updated listings and master tapes. 

Updated documentation. 

User advice. 
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TAfti fXXXVIII . COMPUTER PROGRAMMING COST FACTORS 


FACTOR 


*IMPACTON 
INDICATED RESOURCE 


REFERENCE 




OTHER 


(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


HANDBOOK 
PAGE 


DESCRIPTIVE 


ANALYTICAL 


Requirement G 














X. - Number of organizational users 


+ 


+ 


+ 




63 




Xc - Number of ADP centers 


+ 


+ 


+ 








Xr - Complexity of program system 
interface 


+ 


+ 


+ 




63 




Xq - On-Line requirements 


+ 








63 




X«o - Complexity of system interface 
^ with other systems 


+ 


+ 


+ 




63 




Xni - Number of functions in the 
system 


+ 


+ 


+ 








Xoc - Number of system components 


+ 


+ 


+ 








Xo^ - Number of system components 
^ not off-the-shelf 


+ 


+ 


+ 








Xq^ - Output volume 


+ 


+ 


+ 






23 


^qli " ^P^* volume 


+ 


+ 


+ 






23 


Program Design & Production 














X^^ - Total object instructions 
delivered 


+ 


+ 


+ 




63 




X^ Q - Number of words in the data 
^^ base 


+ 


+ 


+ 




















*NOTE: IMPACT IS INDICATED BY: 


+ = VARIES DIRECTLY 

- = VARIES INVERSELY 

NO SIGN = NO PRESUMED DIRECTION 


(FOR DISCUSSION OF CODING, SEE GLOSSARY D) 
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TARI F XXXVIII : COMPUTER PROGRAMMING COST FACTORS rCONT'D.l 


FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 


♦IMPACT ON 
INDICATED RESOURCE 


REFERENCE 


HANDBOOK 
PAGE 


OTHER 


MAN 
MONTHS 


COMP. 
HOURS 


ELAPSED 
TIME 


DESCRIPTIVE 


ANALYTICAL 


X^ - Nxunber of classes of items 
•^ in data base 


+ 


+ 


+ 








X^ - Nunber of input message types 


+ 


+ 


+ 






23 


Xp^ - Number of output message types 


+ 


+ 


+ 






23 


Xr^r^ - Number of input variables 


+ 


+ 


+ 






23 


Xp^ - Number of output variables 


+ 


+ 


+ 








XpK - Number of words in tables and 
constants not in data base 


+ 


+ 


+ 








X-^r - Average operate time 


+ 


+ 


+ 








X^„ - Frequency of operation 


+ 


+ 


+ 


Ilk 






X. o - Programming language 








Ilk 


55 




X. c , - Internal documentation written 


+ 


+ 


+ 








Y 

ii5.2 - Internal documentation 
available 


- 


- 


- 








X, ^ - External documentation 
required 


+ 


+ 


+ 








X. „ , - Total number of external 
'• document types written 


+ 


+ 


+ 








X. „ P - Total number of internal 
' * document types available 


- 


- 


- 








X, ^ - Total number of internal 
''^ document types written 


- 


- 


- 




63 




X^^Q - Type of program 














Data Processing Equipment 














Xc, - First program on computer 


+ 


+ 


+ 








X ^ - ADP ccniponents developed 
-^-^ concurrently 


+ 


+ 


+ 








Xj.. - Special display equipment 


- 


+ 


+ 









111 



TABLEJQQOail: 



COMPUTER PROGRAMMING COST FACTORS (CONT'D.) 



FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 



♦IMPACT ON 
INDICATED RESOURCE 



MAN 
MONTHS 



COMP. 
HOURS 



ELAPSED 
TIME 



REFERENCE 



HANDBOOK 
PAGE 



OTHER 



DESCRIPTIVE ANALYTICAL 



Personnel 



"61 

h2 



^63 



"64 



^65 

he 



"87 
^92 



- Percent senior programmers 

- Average programmer experience 
with language 

- Average programmer experience 
with application 

- Percent programmers partici- 
pating in design 

- Personnel continuity 

- Maximum number of programmers 
assigned 

- Percent senior analysts 

- Personnel turnover 



Development Environment 



^67 
^69 

V 
^9 



- Lack of management procedures 

- Customer inexperience 

- Computer operated by agency 
other than developer 

- Program developed at site 
other than the operational 
installation 

- Closed or open shop operation 

- Security classification level 

- Number of sources of system 
information 

- Accessibility of system 
inf ormat ion 



Xoo - Quality of resource documents 






+ 



+ 



114 



63, 36 



63 
63 



57 
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TABLeooaaii-: 



COMPUTER PROGRAMMING COST FACTORS (CONT'D.; 



FACTOR 

(FOR DEFINITION & CODING, 
SEE GLOSSARY A) 



♦IMPACT ON 
INDICATED RESOURCE 



MAN 
MONTHS 



COMP. 
HOURS 



ELAPSED 
TIME 



REFERENCE 



HANDBOOK 
PAGE 



OTHER 



DESCRIPTIVE ANALYTICAL 



^91 



- Availability of special tools 

- Degree of standardization 
in policy and procedures 

- Number of official reviews of 
documents 



63 
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ADDITIONAL COMMENTS ON SELECTED COST FACTORS 



37 



Frequency of Operation ^ The primary rationale for believing that the 
frequency of operation affects maintenance programming resources is that 
(a) a high-frequency operation offers incentive to improve the efficiency 
of the program, and (b) there is a greater probability of discovering 
errors during production runs. 



X. p - Programming Language , A major claim made for procedure-oriented 

languages, as compared with assembly languages, is that they reduce 
significantly the amount of effort needed for program production and 
maintenance. The basis for this claim is twofold: (a) a program 
written in procedure language is shorter (contains fewer steps) than 
an equivalent program written in an assembly language, and (b) a program 
written in a procedure language is easier to modify than one written in 
assembly language (55) • 



'-66 



Maximum Number of Programmers Assigned * In the program maintenance step, 
a larger number of programmers may be assigned than actually required at 
any arbitrary point in time. The reason for this would be to insure an 
adequate response to emergency situations; that is, if the nature of a 
program was such that the cost of a delay in correcting an error may far 
exceed the cost of extra program maintenance personnel, added personnel 
may be used as a form of protection against this risk (63) wages are 
invested to buy short elapsed time during troubles. 



On balance, larger prograimning organizations, if properly managed, need 
not be less efficient than their smaller counterparts (38) • 
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GLOSSARY A 
DEFINITIONS OF VARIABLES 



This glossary contains the definitions for the resource costs and the variables 
listed in the forms for Computer Programming Cost Factors. There are three 
kinds of veuriables defined — first, the dependent vBriables or resource costs, 
next, the independent variables or cost factors that originated with the 
statistical analyses for the Computer Program Design, Code, and Test activity, 
emd then new independent variables to describe further the cost influences 
on activities in the programming process* 

1. Independent Variables (Resources) 

Y^ - Total Number of Man Months including first-line supervision, for 
the programming step \inder consideration; does not include the 
costs of any associated executive or utility program. 

Yp - Total Number of Computer Hours used by all developmental computers 
during the programming process step. This includes test time only, 
not production time on line runs. 

Y« - Months Elapsed , the niimber of months elapsed between the start 

date of the programming process step and the completion date for 
the step. 

2. Dependent Variables (Cost Factors ) ♦ The variables defined below (ranging 
from X-j_ through X^™) were first identified in the statistical analyses that 
led to the niimerical res\ilts (Pleuining Factors emd Estimating Equations) in 
Section V on Computer Program Design, Code, and Test. In some cases, we have 
generalized the original definitions to extend their xise to the other five 
activities in computer programming and also in some cases the names of the 
factors have been changed. To help the readers who may have read earlier 
reports from the Programming Management Project, we have cited reference (20), 
TM-3026, "Current Results from the Analysis of Cost Data for Computer 
Programming," 26 July I966, the most recent Project report, to show the 
relationship between any of the changed titles and definitions and their 
earlier forms. The coding used to quantify the variables for the earlier 
analysis is assumed to be suitable for an initial emalysis of the other 
activities. When one independent variable is related to euiother, a note 

has been inserted in the definition. 
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X- - Vagueness of Design Requirements Definition, measures the precise- 
ness of the definition of the programming job. Coded: direct 
trajislation of (simple) manual tasks to automatic functions « 0; 
only a few new functions automated, with requirements relatively 
clear and precise » 1; many new functions to be automated, require- 
ments relatively clear ajid precise = 2; majiy ill-defined and 
unstructured functions to be automated « 3 (same as "Design 
Characteristics" in (20) • 

Xp - Innovation Required , measures whether a significant portion of 
the design ajid production of the program data point involved 
applications or programming techniques that were new to the 
personnel involved • Coded: yes - 1; no = 0. 

X^ - Lack of Knowledge of Operational Requirements, measures how well 
the operational requirements of the program data point were known 
ajid documented. Coded: in great detail = 0; in broad outline « 1; 
only vaguely = 2. See elIso Xi^c 2» 

Xi - Number of Orgajiizational Users , represents the number of orgajil- 
zational interfaces with which the program data point must 
communicate. This variable has a minimum value of l.O. 

Xc- - Niimber of ADP Centers, represents the number of computer-based 
locations with which the program data point must communicate. 
This variable has a minimum value of 1.0. 

X/r - Complexity of Program System Interface, measures the interaction 

between the program data point and other programs or l/o equipment. 
Examples: if the program were a compiler, the measure woiild 
represent the degree of programmer effort devoted to data (e.g., 
source statement) hajidling, interpretation, ajid formatting, but 
not effort spent on internal logic of program; if program involved 
remote I/O devices, measure would represent the degree of pro- 
grammer effort devoted to making the data from the l/o devices 
acceptable to basic program. Coded: more than 50 percent of 
design effort devoted to data transfer problems to or from the 
program data point = 2; between 10 percent and 50 percent effort 
to data transfer problems = 1; less than 10 percent = (same 
as "Complexity of Communication" in (20). 

X^ - Response Time Requirements , measures the time restraints within 
which the operating program must perform. Coded: greater than 
1 day = 0; 2k hours or less = 1; 1 hour or less = 2; real time = 3- 

Xo - Stability of Design , measures the frequency ajid extent of program 
design changes. Coded: initial design carried through without 
chajige = 0; few changes to initial program design = 1; frequent 
chajiges to program design = 2; initial program design almost 
completely revised = 3* See also X^. 
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Xq - On*-Llne Requirements^ examines the operational characteristics 
^ of the program. Coded: no on-line, real-time operation » 0; 

mixture of on-line and off-line operations « 1; mainly on-line, 
real-time operation = 2. 

X^ - Total Object Instructions Delivered, the total number of object 
instructions delivered as part of the program data point. This 
number should include object instructions borrowed and/or reused 
from other programs, which are actually a part of the total 
program listing, as well as those instructions specifically 
written for the program. Coded in thousands. 

X, , - Percent Delivered Object Instructions Reused , the ratio of reused 
object instructions to the total number of object instructions 
delivered. 

X-p - Total Nondelivered Object Instructions Produced , the total number 
of object instructions written, but not delivered as part of the 
finished package. This should include all utility and support 
programs which had to be written (but not delivered) in order 
to produce the desired program. Coded in thousands. 

X, - - Total Source Instructions Written , the total number of source 
instructions (assembly and procedure-oriented) written. Coded 
in thousands. 

X-r - Percent Source Instructions Written in POL , the ratio of 

procedure-oriented (POL) source language statements written 
to the total number of source instructions written. 

X- - Percent of Total Object Instructions Discarded , the ratio of 
the number of generated object instructions discarded due to 
errors and design changes, to the total number of object 
instructions generated. 

X-./r - Percent of Total Source Instructions Discarded , the ratio of the 
number of source instructions discarded due to errors and design 
changes, to the total number of source instructions written. 

X-„ - Number of Conditional Branches, the number of machine language 
(object code) statements that are conditional branches or jumps. 
Coded In thousands. 

X-Q - Number of Words in the Data Base , number of words in the subset 
of tables that describe the environment of the problem to be 
solved by the program data point and/or files to be processed. 
Coded in thousands. 
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X,Q - Nxamber of Classes of Items in the Data Base , classes of items 
•^ (i.e., variables) being categories such as names, salaries, 
states, or any other characteristic of information for which 
there are many items or entries. 

X^ - Number of Input Message Types , the number of different records 
^ (groups of fields of infonnation treated as a unit) used in input 

to the program data point. 

Xp, - Nxamber of Output Message Types, the number of different records 
■ (groups of fields of information treated as a unit) processed as 

outputs from the program data point. The number of distinct 
report formats. 

1 

Xpp - Number of Input Variables , the number of different quantities or 

fields of information that appear as inputs to the program data 

point and which assume any value in a set of numbers or symbols. 

' Xp-3 - Number of Output Variables , the number of different quantities or 

fields of information that appear as outputs from the program data 
point. 

I Xpi - Number of Words in Tables, and Constants not in Data Base , includes 

, all words in tables and constants not in the data base but con- 

I sidered to be describing the problem environment. See variable X-q. 

^Xpc " Percent Clerical Instructions , the part of the instructions 
' comprising the program data point that are used for bookkeeping, 

sorting, searching, and file maintenance. 

•^Xp/- - Percent Mathematical Instructions , the part of the instructions 

i comprising the program data point that are used for evaluating 

and computing algebraic, mathematical, geometric, and trigonometric 
formulas . 



*Note: Variables X25-X2Q characterize the computer program by percentage 
of instructions used for various levels of data processing. The 
levels are a gross way to indicate types of data processing from 
a programmer's point of view. Variables X0Q-X05 stem from a 
different, less precise way to characterize a computer program. 
Percentage of functions, the basis for this classification, is 
oriented toward a user's description of a computer program. For 
example, percent generation (X^^) refers to the proportion of 
program functions devoted to the creation of information from 
data by the application of some logical process. 
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*Xp„ - Percent Input /Output Instructions^ the part of the instructions 
comprising the program data point that are used for performing 
data acceptance and output formatting. 

*Xpo - Percent Logical Control Instructions ^ the part of the instruc- 
tions comprising the program data point that are used for the 
sequencing of operations according to orders, priorities, or 
timing requirements. 

*XpQ - Percent Self-Checking Instructions , the part of the instructions 
'^ comprising the program data point that are used for monitoring 

programs which detect, report, and in some cases attempt to 
correct errors* 

*X-^ - Percent Information Storage and Retrieval Functions , that part of 
the program data point devoted to the storage and retrieval of 
data. 

•>«-X-^ - Percent Data Acqixisition and Display Function , that part of the 
program data point devoted to data acqiiisition and display 
procedures. 

•>«-X-p - Percent Control or Regulation Function , that part of the program 

data point devoted to the control or regulation of data and process 
flow. 

•Jt-X^- - Percent Decision-Making Functions , that peirt of the program data 
point devoted to logical alternatives and choices in the process 
flow. 

•>«-X-j - Percent Transformation Functions, that part of the program data 
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point devoted to the transformation and/or reformatting of data. 



•>«-X-c - Percent Generation Functions , that part of the program data point 
devoted to generating the reqiiired outputs in the program. 

X-/- - Average Operate Time, the actual average time lapse for processing 
the average data load with the operational program data point. 
Coded: not applicable (e.g., utility routines) = 0; less than 
15 minutes = 1; 15 minutes to 1 hour = 2; greater than 1 hour = 3* 

X„ - Frequency of Operation , the average number of times of program 
data point operation. Coded: not applicable = 0; less than 
l/month = 1; more than l/month and less than l/week = 2; more 
than l/week and less than l/day = 3; daily = ^; utility or 
on-line (including compilers) = 5* 



•**-See note on previous page. 
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X-Q - Insufficient Memory , if a memory size was a factor to be considered 
in the programming of the problem, enter 1; if no memory constraint 
existed, enter 0. 

X-Q - Insufficient l/o Capacity , if the lack of l/o channels imposed 
programming difficulty, enter 1; if not, enter 0. 

Xx^ - Stringent Timing Requirements , if strict schedule restrictions 
were imposed on the program development effort, enter 1; if no 
tight schedules existed, enter 0. 

X2^^ - Number of Subprograms , a subprogram being a veil defined set of 
instructions to perform a mathematical or logical operation 
within a specific program data point. 

Xhp - Programming Language , i.e., whether the source language consisted 

of assembly or machine-oriented langiiage instructions or procedure- 
oriented language statements. Coded MOL = 1; POL = 0. 

Xk- - POL Expansion Ratio , measures the rate of expansion between POL 
source statements and the corresponding generated machine code. 

Xkk - Support Program Availability , that is, the extent to which support 
software needed to produce the program data point was available. 
The extent to which support software had to be written for a given 
data point is measured by variable X-^p' I" general terms, covering 
all steps in the programming process, this variable is related to 

Xlc " Internal Documentation^ measures the number of pages of documenta- 
tion distributed within the programming organization, such as 
program data point specifications, design specifications, etc. 

^Us 1 " Documents written during a programming step. 

Xx^c p " Documents available (extent of documentation) from previous step. 

Xi ^ - External Documentation , measures the number of pages of documenta- 
tion written for or distributed to customers, such as program 
design specifications and operating instructions, during indicated 
step. 

Xx^rj - Total Number of Document [gypes, includes all distinct document 
types, e.g., flow charts, operating instructions, and program 
design specifications. 

Xx^j •. - Total number of external document types written during a 
programming step. 
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X^„ p " Total number of Internal document types available (i.e., the 
extent of internal documentation) from previous step. 

X|^„ ^ - Total number of internal docimient types written during a 
programming step. 

X^^Q - ^lype of Program , such as business, scientific, utility, or other 
application. 

^. 



mutually exclusive 



^48 2 " Scientific 
^48.3 " Utility 

Note: Variables Xi^^-^ through Xi^^i^ coded as a mutually exclusive 
binary variable, i.e., if a program data point is a business 
program, that application is scored as 1, while the remain- 
ing applications are scored as 0. Obviously, there are 
many possible ways of classifying programs. 

^liA s " Stand- Alone. Coded: yes » 1; no = 0. 

Xlq - Compiler or Assembler Used , such as the various versions of COBOL, 
^ FAP, FORTRAN, GAP, etc. 

Xc^ - Developmental Computer Used , the make and model of the major 
computer. 

X - First Program on Computer , whether it is a new machine or new to 

'^ the installation and to the programmers writing the program. 
Coded: new = 1; not new = 0. 

Xcp - Average Turnaround Time , measures the time lags (in minutes, 
hours, days) from the time a programmer submits a run to the 
time it is returned to him. Coded: less than 2 hours = 0; 
2 to 11 ho\irs = 1; 12 to 2U hours = 2; greater than 2U hours = 3. 

Xco - ADP Components Developed Concurrently , hardware components are 
to be developed concurrently with the program data point. 
Coded: yes « 1; no = 0. 

XcK - Special Display Equiiament , use of such graphic display as CRT 
scopes, etc. Coded: yes =1; no = 0. 

Xcc " Core Capacity , number of words in the main memory of the major 
computer used in program data point development. 
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Xc/r - Rsindom Access Device Used ^ such as drum, disc, etc. Coded: 
yes =1; no = 0, if any such equipment was used. 

Xc.„ - Number of Bits per Word, number of bits per memory word in the 
major computer used in development. 

Xco - Memory Access Time , measures the time required to read and 
restore a memory word. Coded: microseconds x 10. 

XcQ - Machine Add Time , the time required to acquire and execute one 
fixed-point add instruction. Coded in microseconds. 

X/^p^ - Computer Cost , monthly rental or equivalent purchase price. 
Coded: large-scale computer ($750,000 and up) = 0; medium- 
scale computer ($100,000 to $7^9,000) = 1; smetll-scale computer 
(under $100,000) = 2. 

X/-, - Percent Senior Programmers , the ratio of the total number of 
senior and systems, programmers, to the total number of pro- 
grammers assigned to the project, based on the following 
position descriptions: 

Position Description 

Coder Writes machine language instructions from flow 
charts. Helps prepare flow charts and test 
programs . 

Programmer Develops programs to solve veil-defined problems. 
Prepares flov charts, writes instructions, tests 
programs, modifies established computer programs. 

Senior Conceives, develops and improves large, complex 
Programmer computer programs, e.g., automatic programming 

routines. Improves efficiency of existing 

programs . 

System Formulates and plans new program system applications 
Programmer Keeps abreast of related economic disciplines and 
new information processing technology. Is highly 
creative in designing and developing major computer 
program systems. 

X/^p - Average Programmer Experience with Language , the average nimber 
of months of experience that the assigned programming staff has 
had with the language used in coding the program data point. 
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X^- - Average Programmer Experience vlth Application ^ the average number 
of years' experience that the assigned programming staff has had 
with the application represented by the program data point. 

X^l^ - Percent Programmers Participating in Program Design , the ratio of 
programmers participating in the design of the program data point 
to the total member of programmers assigned to the program data 
point development, 

X^c - Personnel Continixity , the number of personnel working for the 

duration of the project divided by the maximum number assigned at 
any one time. This variable measures the degree of fluctuation 
in the size of staff; it is related to X^p. 

Xx-/- - Maximum Number of Progreimmers , the maximum number of programmers 
assigned to the program data point at any one time. 

X/Tr. - Lack of Management Procedures, measures the use of eleven manage- 
ment procedures, such as the use of PERT charts, evaluation of 
program design changes, dissemination of error-detection and 
-correction procedures, contingency plans for computer overload, 
etc. Coded: the number of "no*' replies to the eleven procedures 
(see reference (4o)). 

X/-n - Number of Agencies Concurring in Design , the number of different 
organizations or agencies that have to sign-off on proposed 
program data point design. 

Xx-Q - Customer Inexperience, measures customer knowledge and experience 
in developing EDP systems. Coded: extensive = 0; limited = 1; 
none = 2. 

^n " ^QJ^P^"^^^ Operated by Agency Other than Program Developer . 
Coded: yes = 1; no = 0. 

^1 " Program Developed at Site other than the Operational Inst6illation . 
Coded: yes =1; no = 0. 
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Different Computers for Programming and Operation , refers to 
whether a different model of computer was used for program 
development and operation. Coded: yes =1; no = 0. 

Closed or Open Shop Operation , indicates the type of installation 
in which the program data point was developed. Coded: open shop = 
0; closed shop = 1. 



X«L - Number of Locations for Program Data Point Development , measures 

the number of different geographical locations at which the program 
data point was developed. 
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X c - Number of Mein Trips , measiires the number of man trips required 
for conciirrence dixring design, code, and test phases of program 
data point developments 

• Program Data Point Developed by Military Organization . Coded: 
yes =1; no = 0. 
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X^„ - Program Data Point Developed on Time-Shared Computer . Coded: 
yes = 1; no = 0. 

Note: Variables Xyg through Xq-|_ are factors that have not been used in 
any numerical analysis. Therefore, they represent hypotheses 
which have not been investigated in terms of data; they may not 
be as precisely defined as the previous 77 factors. 

X^o - Complexity of System Interface vith Other Systems , would measure 

the dependence of the subject system, i.e., that under development, 
upon other data processing (information) systems in terms of 
providing inputs to them or accepting outputs from them. The 
complexity would be a function of the number of messages from 
or to each of the related systems, as well as the degree of 
dependence as it relates to performance of specific organizational 
missions, for example, messages to and from a computer in a missile 
for guidance and control would represent a high degree of dependence 
in executing an organizational mission. Related to X/-. 

X„Q - Security Classification Level , would measure security classifi- 
cation level (e.g.. Company Private, Confidential, Secret, 
Top Secret). The hypothesis is that the higher the level of 
classification, the more costly (and time-consimiing) the work 
in terms of obtaining personnel and documents containing 
essential information, safeguarding the current results of 
the work and trsinsmitting these results to other cognizant 
personnel. 

Xnp, - Niimber of Sources of System Information , would raeasiire the 

dispersion of various kinds of information needed to conduct 
the system analysis and design work, as well as other activities 
in computer program development. The hypothesis is that if 
there are a large number of sources, then the cost of the work 
would increase; for example, interviewing a large number of 
personnel or secixring documents from different libraries or 
information sources. 
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Xq - Accessibility of System Information, is closely related to Xqq, 
that is a measure of the effort with which information may be 
acquired, (in this sense, it is also related to Xyg, security 
classification.) The hypothesis here is that if information, 
such as docioments, is easy to obtain, then the cost of system 
analysis and design work as well as the conduct of other 
activities in computer prograimning would be reduced. 

Xop - Degree of System Change Expected During Development , would 

measure the expected change, for example, in interfacing systems 
and in organizational mission reflected in the system functions 
during the development period. The hypothesis is that the 
greater the amount of change expected, the higher the cost will 
be during the entire process of computer program development, 
particularly during system aneilysis and design; this is 
because effort is spent making allowEuices for contingencies. 
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• Degree of System Change Expected During System Operations . 
Closely related to Xop^ this factor would be a measure of the 
amount of flexibility and versatility that must be designed 
into the proposed system. Therefore, the hypothesis is that 
additional cost woiild be incurred with a higher degree of 
expected change and would be reflected throughout the computer 
programming process. 

Xqx - Number of Functions in the System, would measure the specific 
parts of the organizational mission that would be changed by 
the development of the new system which would provide automatic 
data processing for all or parts of these functions. The 
hypothesis is that the greater the number of functions, the 
higher the cost in all of the computer program development 
activities. 

Xqc - Number of System Components , a count of the various system 

elements that are part of information processing or controlled 
by it. For example, sensors, communication links, communication 
terminals, weapons, production levels. The hypothesis is that 
the greater the number of components, the greater the cost. 

Xn^ - Number of System Components — Not Off-the-Shelf , measures the 
newness or innovation required in all (ADP and other) system 
components. It is closely related to X2> Xco, and Xgc. The 
hypothesis is that as the number of components that must be 
developed along with computer programs increases so does the 
cost of computer program development. 
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Xn„ - Percent Senior Analysts (related to Xg,, percent senior programmers); 
is a measure of the extent to which experience and proficiency are 
applied to the job. One hypothesis is that the higher the cost of 
the initial investment, particularly in system design and analysis, 
the lover the total cost of the system over its lifetime. In the 
systems analysis step, the hypothesis is that more proficient 
analysts do the job with fewer resoxirce units. 

Xno - Q^ality of Resource Documents , would be a measure of the accuracy 
consistency, completeness, freshness (up to date), and ease of 
understanding for the documents used as sources of information 
in doing computer program development, particularly the system 
analysis and design work. The hypothesis is that the better the 
quality, the lover the cost. 

X^ - The Availability of Special Tools ; for example, computer programs 
for simulation. This factor is a measure of the accessibility and 
reliability of tools that could be used to facilitate the computer 
program development, particularly system analysis and design. The 
hypothesis is that if such tools are available, they will help 
speed the work and reduce the cost. This factor is related to, 
but is an expansion of, X. . . 

Xqo " Degree of Standardization in Policy and Procedures , measures how 
^ hov much formal guidance has been introduced for controlling the 
process of computer program development (possibly as part of the 
development of a larger system). The hypothesis is that the more 
such standards are introduced, the higher the cost will be for 
the initial development process (since these actually are a kind 
of constraint), but the lover the cost of the total system over 
its lifetime. 

Xq, - Number of Official Reviews of Dociiments , may be a direct count 
^ of the formal inspections during the process of computer program 
development. The hypothesis is that the greater the number of 
these reviews, the higher the cost of computer programming; but 
the lower the total cost of system maintenance over its lifetime. 
This factor is related to X/-o* 

X^p - Personnel Turnover , the number of personnel on the project 

replaced per time period, divided by the total staff. Turnover 
measures the instability of the staff assigned to the project. 
It is related to Xz-^; but X^prefers to the replacement of 
individual project members. 

Xq^ - Output Voliime , the expected amount of output from the user^s 
operation of the program data point. Measured in average 
characters, exclusive of blanks, per month. 
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Xqj^ - Input Volimie, the expected amount of input for user's operation 
^ of the program data point. Measured in average number of 
characters per month. 
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GLOSSARY B 
DEFINITION OF SYMBOLS 



This glossary contains the definitions of symbols that occur in the text — 
mainly in the Planning Factors and Estimating Equations in Section V, Computer 
Program Design, Code, and Test. 

r = multiple correlation coefficient 

This statistic is a measure of the interrelationship among all the 
independent variables, (cost factors), and the dependent variable 
(cost) 

2 

r = coefficient of determination 

explained sum of squares 

total sum of squares 
This statistic, the square of the multiple correlation coefficient 
shows hov much of the variation in the dependent variable being pre- 
dicted is explained by the estimating relationship, e.g., an equation. 
The range is zero to one; the higher the value the better the equation, 

N = total number of data points in a given sample 

P = standardized regression coefficient (3) 

This statistic is a measure of the relative strength of a particular 
independent variable (cost factor) that appears in an equation 

o = standard deviation of a variable 

This statistic is a measure of the spread or variation in a variable. 
In this Handbook, the standard deviation is usually shown for a cost 
or dependent variable. 

Q_ = standard error of the estimate of a regression equation 

This statistic, a measure of the estimating precision (accuracy) of 
the estimating equation, determines the confidence intervals such as 
the Stanine bands used in the text. The smaller the value the better 
the equation. 
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GLOSSARY C 
USE OF TEEOC 



This glossary describes the ways particuleu: terms, usually familiar words, 
are used in this Handbook. 

1. Program Data Point . Each member of the sample of computer programs 
representing a peurticular program for which sepeurate and distinct data axe 
available. To qualify as a data point, the data m\ist be for a programming 
effort that resulted in (a) the smallest number of instructions developed 
for a user with specific computer programming requirements, and (b) a program 
or product capable of operating in the computer as a single entity or package. 

2- Open Shop . A computer facility with operating procedures that allow 
programmers to have direct access to the computer for the purpose of running 
and debugging programs . 

3. Closed Shop . A computer facility with operating procedures that require 
programmers to submit all Jobs to a second pjarty. The programmer has no 
control over the scheduling of Jobs, or the physical operation of the heu:dweu:e. 

U. Resource Unit . A single quantity of any of the resources used in the 
computer programming process. The three resource units considered in this 
report eu:e the man month, the computer hour, and the month of elapsed time. 

5. Computer Program . The words computer program may mean several different 
things, depending upon the context, e.g., a computer program system that 
consists of an interrelated collection of programs designed to work together 
to perform certain data processing functions, a part of such a program system 
corresponding to a single function, or a single unit that performs by itself 
and corresponds to a single function. Throughout the text, qualifiers have 
been used to establish a context that helps the reader interpret the meaning 
appropriately. Also, in some places, for clarity, synonyms are used for the 
words computer program when they refer to part of a computer program system, 
e.g., subprogram , run , individual program , routine . 

6. System . This term is used to refer to either a computer program system 
or an information (processing) system. Qualifiers eu:e used in context to 
help the reader interpret the term correctly. For example, in the sentence 
".. .determining. . .requirements for improved information processing and 
planning of a system plus a set of computer programs...." the system is 

an information processing system . 
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7. Information System * This term, a short form of information processing 
system , is used to put computer programming work in a context. In its 
broadest sense, an information system includes men, machines, procedures, 
commiinications, and the computer program that work together at one or more 
locations to process information automatically and manually to help fulfill 
one or more missions in an orgemization. The automatic parts of an informa- 
tion system could be called an automatic data processing system * 

8. Manual. This term, used as an adjective, means nonautomatic. 

9. Data Processing . This terms is used as a synonym for automatic data 
processing . 

10. Data Processing Project* This term refers to a specific organized 
effort to develop (or modify) a data processing system and is sometimes used 
synonymously with data processing system . The development work for a data 
processing project includes computer programming as one major type of work. 

11 . Function . This term refers to an integral \init of data processing 
that corresponds to a user's information processing responsibility and 
that is usually described in terms understemdable to both user emd 
information processing personnel. 
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GLOSSARY D 

DEFINITION FOR THE CODE TO RATE THE 
IMPACT OF COST FACTORS 



This glossary contains definitions of the code used to rate the influence or 
impact of a particular factor upon a particular resource cost. In the text, 
these ratings are found in the forms for conputer progranuning cost factors in 
each of the sections (ill through VIII ) on the progranuning activities. Most 
variables (cost factors) listed for each step in the programming process will 
have a plus (+) or minus (-) sign to indicate the presumed direction of impact 
of the variable on the indicated resource. In addition, the numeric indicator 
for correlation could be determined only when empirical data were available; 
these indicators appear only in the section on Computer Program Design, Code, 
and Test. Also, some cost factors in this section were given an additional 
rating to explain why the variable is included. 

1. Correlation. The following numerical ratings indicate the degree of 
correlation of the independent variable with the Indicated resource: 

k = High correlation: statistically significant at the 1% level 
(r ^ .20 for N = 169) 

3 = Moderate correlation: statistically significant at the 55^ level 
(.20 > r i .15 for N = I69) 

2 = Low correlation (r < .I5) 

1 = Indeterminate: present data base not adequate for statement of 
degree of correlations for this variable; or correlation has an 
illogical sign for some subsamples. 

Reference 6 contains the specific correlation coefficients for the variables 
rated numerically in this Handbook. In general, the definitions for the 
variables that appear in Glossary A were so constructed that the sign of the 
correlation coefficient would be expected to be positive. Thus, X^q- -Computer 
Cost, was coded for large-, 1 for medium-, and 2 for small-scale computers, 
since an increase in the variable so coded would probably result in an increase 
in the resource under consideration. 
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2. Signs , The direction of relationship is indicated by: 

+ = Resource varies, or shoiild be expected to vary, directly with variable; 
an increase in the value of the variable, as defined in Glossary A, 
results in an increase in the amount of the resource. 

- = Resource varies, or should be expected to vary, inversely with variable; 
an increase in the value of the variable as defined in Glossary A, 
results in a decrease in the amount of the resource. 

The lack of a sign code indicates that the direction of variation of 
the resource with the presence of the factor is not clear, or should 
not be presumed. Sign codes will always be present with correlation 
codes i^-, 3^ or 2; but sign codes may be absent when the correlation 
code is 1. When only a sign code appears, the variable has not been 
analyzed statistically. 

3- Other Considerations . In addition, the following rating codes appear only 
for cost factors in the design, code, and test step in the programming process: 

A = This cost factor, taken independently, has high correlation with the 
indicated resource; but this factor does not appear in the estimating 
equations, because it has been replaced by another superior variable 
with which it is closely related, i.e., correlates. 

B = Analysis to date has not demonstrated the impact of this variable for 
the indicated resource. However, the variable maintains strong 
intuitive appeal, and merits further research. 
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